The controllers in your environment are upgraded at the cluster level. If an upgrade is available for a controller node, an upgrade link appears in the NSX Manager.

Upgrade the controllers during a maintenance window.

The NSX Controller upgrade causes an upgrade file to be downloaded to each controller node. The controllers are upgraded one at a time. While an upgrade is in progress, the Upgrade Available link is not clickable, and API calls to upgrade the controller cluster are blocked until the upgrade is complete.

Important: In NSX 6.3.3 the underlying operating system of the NSX Controller changes. When the operating system of the old and new controllers is the same, the controller upgrades is an in-place software upgrade. When you upgrade from NSX 6.3.2 or earlier to NSX 6.3.3 or later, the operating systems are different and so an in-place upgrade cannot be performed. Instead the existing controllers are deleted one at a time, and new Photon OS-based controllers are deployed using the same IP addresses.

When the controllers are deleted, any associated DRS anti-affinity rules are deleted. You must create new anti-affinity rules in vCenter to prevent the new controller VMs from residing on the same host.

If you deploy new controllers before the controller upgrade has started, they are deployed as the old version. If you deploy new controllers after the controller upgrade has started, they are deployed as the new version. Controller nodes must be the same version to join a cluster.

Prerequisites

  • Ensure that all the controllers are in the normal state. Upgrading is not possible when one or more of the controllers are in the disconnected state. To reconnect a disconnected controller, try resetting the controller virtual appliance. In the Hosts and Clusters view, right-click the controller and select Power > Reset. For more information about troubleshooting the NSX Controller cluster, see "NSX Controller Cluster Failures" in the NSX Troubleshooting Guide.

  • A valid NSX Controller cluster contains three controller nodes. Log in to the three controller nodes and run the show control-cluster status command. 

    controller-node# show control-cluster status 
			 
Type                Status                                       Since
--------------------------------------------------------------------------------
Join status:        Join complete                                05/04 02:36:03
Majority status:    Connected to cluster majority                05/19 23:57:23
Restart status:     This controller can be safely restarted      05/19 23:57:12
Cluster ID:         ff3ebaeb-de68-4455-a3ca-4824e31863a8
Node UUID:          ff3ebaeb-de68-4455-a3ca-4824e31863a8

Role                Configured status   Active status
--------------------------------------------------------------------------------
api_provider        enabled             activated
persistence_server  enabled             activated
switch_manager      enabled             activated
logical_manager     enabled             activated
directory_server    enabled             activated
    • For Join status, verify that the controller node is reporting Join Complete.
    • For Majority status, verify that the controller is connected to the cluster majority.
    • For Cluster ID, all the controller nodes in a cluster have the same cluster ID.
    • For Configured status and Active status, verify that the all the controller roles have the status of enabled and activated.
  • Make sure that you understand the operational impact of the NSX Controller upgrade while the upgrade is in progress. See Operational Impacts of NSX Upgrades.
  • The NSX Controller cluster must contain three controller nodes. If it has fewer than three, you must add additional nodes before starting the upgrade. See "Deploy NSX Controller Cluster" in the NSX Installation Guide for steps to add controller nodes.

Procedure

  1. Log in to the vSphere Web Client.
  2. Navigate to Networking & Security > Installation and Upgrade > Management > NSX Managers.
  3. Click Upgrade Available in the Controller Cluster Status column.
    The controllers in your environment are upgraded and rebooted one at a time. After you initiate the upgrade, the system downloads the upgrade file, upgrades each controller, reboots each controller, and updates the upgrade status of each controller.
  4. Monitor the progress of the upgrade.
    • You can view the cluster upgrade progress in the Controller Cluster Status column in Installation and Upgrade > Management > NSX Managers.
    • You can view the upgrade progress of each individual controller node in the Upgrade Status column in Installation and Upgrade > Management > NSX Controller Nodes.

Results

When the upgrade is complete, the Software Version column in the NSX Controller nodes section displays 6.4.buildNumber for each controller. Rerun the show control-cluster status command to make sure that the controllers can create a majority. If the NSX Controller cluster majority is not achieved, review the controller and NSX Manager logs.

The average upgrade time for each upgrade is 6–8 minutes. If the upgrade does not finish within the timeout period (30 minutes), the Upgrade Status column displays Failed. Click Upgrade Available in the NSX Manager section again to resume the upgrade process from the point where it stopped.

If network problems are preventing a successful upgrade within the 30-minute timeout period, work with VMware Support to diagnose and resolve any underlying problems.

If the controller upgrade fails, check connectivity between the controllers and the NSX Manager.

There is a scenario in which the first controller upgrades successfully, and the second controller does not. Assuming you have three controllers in a cluster, the first controller is successfully upgraded to the new version, and the second controller is being upgraded. If the upgrade of the second controller fails, the second controller might be left in a disconnected state. At the same time, the first and third controllers now have two different versions (one upgraded, one not) and are therefore unable to form a majority. At this point, the upgrade cannot be relaunched. To work around this scenario, create another controller. The newly created controller has the newer NSX version (matching controller one) and can form a majority with controller one. At this point, you can relaunch the upgrade procedure. See "Redeploy an NSX Controller" in the NSX Troubleshooting Guide for instructions on creating another controller.