You shut down the components of a VI workload domain that runs virtualized workloads in VMware Cloud Foundation in a specific order to keep components operational by maintaining the necessary infrastructure, networking, and management services as long as possible before shutdown.

You shut down the management components for the VI workload domains before you shut down the components for the management domain.

If the NSX Manager cluster and NSX Edge cluster are shared with other VI workload domains, follow this general order:

  1. Shut down the customer workloads in all VI workload domains that share the NSX-T Data Center instance. Otherwise, all NSX networking services in the customer workloads will be interrupted when you shut down NSX-T Data Center.

  2. Shut down the VI workload domain that runs the shared NSX Edge nodes.

  3. Shut down the other VI workload domains.

Shutdown Order for a VI Workload Domain

Table 1. Shutdown Order for a VI Workload Domain

Shutdown Order

SDDC Component

1

Virtualized customer workloads

2

Site Recovery Manager for the VI workload domain

3

vSphere Replication for the VI workload domain

4

NSX Edge nodes for the VI workload domain *

5

NSX Manager nodes for the VI workload domain *

6

vSphere Cluster Services virtual machines in the VI workload domain *

7

ESXi hosts and VMware vSAN™ in the VI workload domain *

8

vCenter Server for the VI workload domain *

* For information on the shutdown steps, see this article.

Shut Down the NSX Edge Nodes

You begin shutting down the VMware NSX-T™ Data Center infrastructure in the management domain or in a VI workload domain in VMware Cloud Foundation by shutting down the NSX Edge nodes that provide north-south traffic connectivity between the physical data center networks and the NSX SDN networks

Procedure

  1. Log in to vCenter Server for the management or VI workload domain at https://<vcenter_server_fqdn>/ui as [email protected].
  2. In the VMs and templates inventory, expand the tree of workload domain vCenter Server and expand data center for the workload domain.
  3. Right-click an NSX Edge virtual machine for the management domain or VI workload domain and select Power > Shut down Guest OS.
  4. In the confirmation dialog box, click Yes.

    This operation takes several minutes to complete.

  5. Repeat the steps for the remaining NSX Edge nodes for the domain.

Shut Down the NSX Manager Nodes

You continue shutting down the NSX-T Data Center infrastructure in the management domain and a VI workload domain by shutting down the three-node NSX Manager cluster by using the vSphere Client.

Procedure

  1. Log in to the management domain vCenter Server at https://<vcenter_server_fqdn>/ui as [email protected].
  2. In the VMs and templates inventory, expand the management domain vCenter Server tree and expand the management domain data center.
  3. Right-click the primary NSX manager virtual machine and select Power > Shut down Guest OS.
  4. In the confirmation dialog box, click Yes.

    This operation takes several minutes to complete.

  5. Repeat the steps for the remaining NSX Manager virtual machines.

Shut Down the vSphere Cluster Services Virtual Machines

To shut down the vSphere Cluster Services (vCLS) virtual machines in a cluster in the management domain or in a VI workload domain in VMware Cloud Foundation, you put the cluster in retreat mode. The retreat mode triggers clean-up of the vCLS virtual machines.

Procedure

  1. Log in to vCenter Server for the management or VI workload domain at https://<vcenter_server_fqdn>/ui as [email protected].
  2. In the Hosts and clusters inventory, expand the tree of the VI workload domain vCenter Server and expand the data center for the VI workload domain.
  3. Select the cluster on which vCLS must be shut down.
  4. Copy the cluster domain ID domain-c(cluster_domain_id) from the URL of the browser.

    When you navigate to a cluster in the vSphere client, the URL is similar to this one:

    https://<fqdn-of-vCenter-server>/ui/app/cluster;nav=h/urn:vmomi:ClusterComputeResource:domain-c8:eef257af-fa50-455a-af7a-6899324fabe6/summary

    You copy only domain-c8.

  5. In the Host and Clusters inventory, select the vCenter Server instance for the management domain or the VI workload domain and click the Configure tab.
  6. Under Advanced Settings, click the Edit Settings button.
  7. Locate the config.vcls.clusters.domain-c(number).enabled property for the domain cluster ID from Step 4 and set it to false.

    If the property is not present, add it. The entry for the cluster cannot be deleted from the vSphere Client then. However, keeping this entry is not an issue.

  8. Click Save.

Results

The vCLS monitoring service initiates the clean-up of vCLS VMs. If vSphere DRS is activated for the cluster, it stops working and you see an additional warning in the cluster summary. vSphere DRS remains deactivated until vCLS is re-activated on this cluster.

Shut Down vSAN and the ESXi Hosts in a Virtual Infrastructure Workload Domain

You shut down vSAN and the ESXi hosts in a VI workload domain in VMware Cloud Foundation by preparing the vSAN cluster for shutdown, placing each ESXi host in maintenance mode to prevent any virtual machines from being deployed to or started up on the host, and shutting down the host.

If you deployed additional clusters in the VI workload domain, repeat this procedure for all ESXi hosts in each cluster.

Procedure

  1. Log in to the management domain vCenter Server at https://<vcenter_server_fqdn>/ui as [email protected].
  2. In the Hosts and clusters inventory, expand the tree of the VI workload domain vCenter Server and expand the data center for the VI workload domain.
  3. For a vSAN cluster, verify the vSAN health and resynchronization status.
    1. Select the cluster and click the Monitor tab.
    2. In the left pane, navigate to vSAN > Skyline health and verify the status of each vSAN health check category.
    3. In the left pane, under vSAN > Resyncing objects, verify that all synchronization tasks are complete.
  4. If a vSAN cluster has vSphere HA turned on, stop vSphere HA to avoid vSphere HA initiated migrations of virtual machines after vSAN is partitioned during the shutdown process.
    1. Select the vSAN cluster and click the Configure tab.
    2. In the left pane, select Services > vSphere Availability and click the Edit button.
    3. In the Edit Cluster Settings dialog box, disable vSphere HA and click OK.

    This operation takes several minutes to complete.

  5. Turn on SSH on the ESXi hosts in the VI workload domain by using the SoS utility of the SDDC Manager appliance.
    You turn on SSH on the management ESXi hosts before you shut down SDDC Manager.
    1. Log in to the SDDC Manager appliance by using a Secure Shell (SSH) client as vcf.
    2. Switch to the root user by running the su command and entering the root password.
    3. Run this command. 
      /opt/vmware/sddc-support/sos --enable-ssh-esxi --domain domain-name
  6. Log in to the first ESXi host of the cluster in the VI workload domain by using a Secure Shell (SSH) client as root.
  7. For a vSAN cluster, disable vSAN cluster member updates by running the command. 
    esxcfg-advcfg -s 1 /VSAN/IgnoreClusterMemberListUpdates

    The command returns Value of IgnoreClusterMemberListUpdates is 1

  8. On the first host per vSAN cluster, prepare the vSAN cluster for shutdown by running the command. 
    python /usr/lib/vmware/vsan/bin/reboot_helper.py prepare

    The command returns Cluster preparation is done!

  9. Place the ESXi host in maintenance mode by running the command. 
    esxcli system maintenanceMode set -e true -m noAction

    Ensure the prompt comes back after the command is compelte.

  10. Verify that the host is in maintenance mode. 
    esxcli system maintenanceMode get
  11. Repeat 6 to Step 10 on the remaining ESX hosts in the cluster, proceeding to the next host after the operation is complete on the current one.
  12. Log in to the VI workload domain vCenter Server at https://<vcenter_server_fqdn>/ui as [email protected].
  13. In the Hosts and clusters inventory, expand the tree of the VI workload domain vCenter Server and expand the data center for the VI workload domain.
  14. Select the cluster and verify that all ESXis are in Maintenance Mode.
  15. Select all ESX hosts.
  16. Right-click the first ESXi host and select Power > Shut Down
  17. Confirm the shutdown operation on the multiple hosts.
  18. Enter a reason for the shutdown operation and click OK.

Shut Down the vCenter Server Instance in a Virtual Infrastructure Workload Domain

To shut down the vCenter Server instance for a VI workload domain in VMware Cloud Foundation, you shut down the vCenter Server virtual machine by using the vSphere Client.

Procedure

  1. Log in to the management domain vCenter Server at https://<vcenter_server_fqdn>/ui as [email protected].
  2. In the VMs and templates inventory, expand the management domain vCenter Server tree and expand the management domain data center.
  3. Locate the vCenter Server virtual machine for the VI workload domain.
  4. Right-click the virtual machine and select Power > Shut down Guest OS.

    This operation takes several minutes to complete.

  5. In the confirmation dialog box, click Yes.