Stretch a vSAN Cluster in VMware Cloud Foundation

You can stretch a vSAN cluster (ESA or OSA) in the management domain or VI workload domain using a JSON specification and the VMware Cloud Foundation API.

When you stretch a cluster, VMware Cloud Foundation modifies the site disaster tolerance setting for storage policy associated with datastore of that cluster from None - standard cluster to Site mirroring - stretched cluster. This affects all VMs using default datastore policy in that cluster. If you do not want to change the site disaster tolerance setting for specific VMs, apply a different storage policy to those VMs before stretching the cluster.

Prerequisites

• Verify that vCenter Server is operational.
• Verify that you have completed the Planning and Preparation Workbook with the management domain or VI workload domain deployment option included.
• Verify that your environment meets the requirements listed in the Prerequisite Checklist sheet in the Planning and Preparation Workbook.
• Create a network pool for availability zone 2.
• Commission hosts for availability zone 2. See Commission Hosts.
• Ensure that you have enough hosts such that there is an equal number of hosts on each availability zone. This is to ensure that there are sufficient resources in case an availability zone goes down completely.
• Deploy and configure a vSAN witness host. See Deploy and Configure vSAN Witness Host.
• If you are stretching a cluster in a VI workload domain, the default management vSphere cluster must have been stretched.

Note: You cannot stretch a cluster in the following cases:

• The cluster is a vSAN Max cluster.
• The cluster has a vSAN remote datastore mounted on it.
• The cluster shares a vSAN Storage Policy with any other clusters.
• The cluster includes DPU-backed hosts.

Procedure

1. Create a JSON specification in a text editor.
   The following example is for an environment with a single vSphere Distributed Switch. If you have multiple vSphere Distributed Switches, see the VMware Cloud Foundation API Reference Guide for details about creating a JSON specification.

   Note: The ESXi hosts that you are adding to availability zone 2 must use the same vmnic to vSphere Distributed Switch mapping as the existing hosts in availability zone 1.

   {
    "clusterStretchSpec": {
     "hostSpecs": [
      {
       "hostname": "sfo02-w01-esx01.sfo.rainpole.io",
       "hostNetworkSpec": {
        "networkProfileName": "sfo-w01-az2-nsx-np01",
        "vmNics": [
         {
          "id": "vmnic0",
          "uplink": "uplink1",
          "vdsName": "sfo-w01-cl01-vds01"
         },
         {
          "id": "vmnic1",
          "uplink": "uplink2",
          "vdsName": "sfo-w01-cl01-vds01"
         }
        ]
       },
       "id": "<ESXi host 1 ID>",
       "licenseKey": "<license key>"
      },
      {
       "hostname": "sfo02-w01-esx02.sfo.rainpole.io",
       "hostNetworkSpec": {
        "networkProfileName": "sfo-w01-az2-nsx-np01",
        "vmNics": [
         {
          "id": "vmnic0",
          "uplink": "uplink1",
          "vdsName": "sfo-w01-cl01-vds01"
         },
         {
          "id": "vmnic1",
          "uplink": "uplink2",
          "vdsName": "sfo-w01-cl01-vds01"
         }
        ]
       },
       "id": "<ESXi host 2 ID>",
       "licenseKey": "<license key>"
      },
      {
       "hostname": "sfo02-w01-esx03.sfo.rainpole.io",
       "hostNetworkSpec": {
        "networkProfileName": "sfo-w01-az2-nsx-np01",
        "vmNics": [
         {
          "id": "vmnic0",
          "uplink": "uplink1",
          "vdsName": "sfo-w01-cl01-vds01"
         },
         {
          "id": "vmnic1",
          "uplink": "uplink2",
          "vdsName": "sfo-w01-cl01-vds01"
         }
        ]
       },
       "id": "<ESXi host 3 ID>",
       "licenseKey": "<license key>"
      },
      {
       "hostname": "sfo02-w01-esx04.sfo.rainpole.io",
       "hostNetworkSpec": {
        "networkProfileName": "sfo-w01-az2-nsx-np01",
        "vmNics": [
         {
          "id": "vmnic0",
          "uplink": "uplink1",
          "vdsName": "sfo-w01-cl01-vds01"
         },
         {
          "id": "vmnic1",
          "uplink": "uplink2",
          "vdsName": "sfo-w01-cl01-vds01"
         }
        ]
       },
       "id": "<ESXi host 4 ID>",
       "licenseKey": "<license key>"
      }
     ],
     "isEdgeClusterConfiguredForMultiAZ": <true, if the cluster hosts an NSX Edge cluster; false, if the cluster does not host an NSX Edge cluster>,
     "networkSpec": {
      "networkProfiles": [
       {
        "isDefault": false,
        "name": "sfo-w01-az2-nsx-np01",
        "nsxtHostSwitchConfigs": [
         {
          "ipAddressPoolName": "sfo-w01-az2-host-ip-pool01",
          "uplinkProfileName": "sfo-w01-az2-host-uplink-profile01",
          "vdsName": "sfo-w01-cl01-vds01",
          "vdsUplinkToNsxUplink": [
           {
            "nsxUplinkName": "uplink-1",
            "vdsUplinkName": "uplink1"
           },
           {
            "nsxUplinkName": "uplink-2",
            "vdsUplinkName": "uplink2"
           }
          ]
         }
        ]
       }
      ],
      "nsxClusterSpec": {
       "ipAddressPoolsSpec": [
        {
         "description": "WLD01 AZ2 Host TEP Pool",
         "name": "sfo-w01-az2-host-ip-pool01",
         "subnets": [
          {
           "cidr": "172.16.44.0/24",
           "gateway": "172.16.44.253",
           "ipAddressPoolRanges": [
            {
             "end": "172.16.44.200",
             "start": "172.16.44.10"
            }
           ]
          }
         ]
        }
       ],
       "uplinkProfiles": [
        {
         "name": "sfo-w01-az2-host-uplink-profile01",
         "teamings": [
          {
           "activeUplinks": [
            "uplink-1",
            "uplink-2"
           ],
           "name": "DEFAULT",
           "policy": "LOADBALANCE_SRCID",
           "standByUplinks": []
          }
         ],
         "transportVlan": 1644
        }
       ]
      }
     },
     "witnessSpec": {
      "fqdn": "sfo-w01-cl01-vsw01.sfo.rainpole.io",
      "vsanCidr": "172.17.11.0/24",
      "vsanIp": "172.17.11.219"
     },
     "witnessTrafficSharedWithVsanTraffic": false
    }
   }

   Note: Replace the example values in the JSON file with the correct values for your environment.
2. In the navigation pane, click Developer Center > API Explorer.
3. Retrieve and replace the unique IDs for each ESXi host in the JSON specification.
   1. Expand the APIs for managing hosts section, and expand GET /v1/hosts.
   2. In the Status text box, enter UNASSIGNED_USEABLE and click Execute.
   3. In the Response section, click PageOfHost, copy the id element of each host, and replace the respective value in the JSON specification.

      ESXi Host	Value
      ESXi Host 1	ESXi host 1 ID
      ESXi Host 2	ESXi host 2 ID
      ESXi Host 3	ESXi host 3 ID
      ESXi Host 4	ESXi host 4 ID

4. Replace the license key value in JSON specification with valid keys.
5. Retrieve the unique ID for the management cluster.
   1. Expand the APIs for managing clusters section, and expand GET /v1/cluster.
   2. Click Execute.
   3. In the Response section, click PageOfCluster, copy the id element of the management cluster.
      You will need the cluster ID for subsequent steps.
6. Validate the JSON specification file.
   1. Expand the APIs for managing clusters section and expand POST /v1/clusters/{id}/validations.
   2. In the Value text box, enter the unique ID for the management cluster that you retrieved in step 5.
   3. In the clusterUpdateSpec text box, type

      {
      "clusterUpdateSpec": 
      }

   4. Paste the JSON specification.
      For example:

      {
      "clusterUpdateSpec": {
       "clusterStretchSpec": {
        "hostSpecs": [
         {
          "hostname": "sfo02-w01-esx01.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 1 ID>",
          "licenseKey": "<license key>"
         },
         {
          "hostname": "sfo02-w01-esx02.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 2 ID>",
          "licenseKey": "<license key>"
         },
         {
          "hostname": "sfo02-w01-esx03.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 3 ID>",
          "licenseKey": "<license key>"
         },
         {
          "hostname": "sfo02-w01-esx04.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 4 ID>",
          "licenseKey": "<license key>"
         }
        ],
        "isEdgeClusterConfiguredForMultiAZ": <true, if the cluster hosts an NSX Edge cluster; false, if the cluster does not host an NSX Edge cluster>,
        "networkSpec": {
         "networkProfiles": [
          {
           "isDefault": false,
           "name": "sfo-w01-az2-nsx-np01",
           "nsxtHostSwitchConfigs": [
            {
             "ipAddressPoolName": "sfo-w01-az2-host-ip-pool01",
             "uplinkProfileName": "sfo-w01-az2-host-uplink-profile01",
             "vdsName": "sfo-w01-cl01-vds01",
             "vdsUplinkToNsxUplink": [
              {
               "nsxUplinkName": "uplink-1",
               "vdsUplinkName": "uplink1"
              },
              {
               "nsxUplinkName": "uplink-2",
               "vdsUplinkName": "uplink2"
              }
             ]
            }
           ]
          }
         ],
         "nsxClusterSpec": {
          "ipAddressPoolsSpec": [
           {
            "description": "WLD01 AZ2 Host TEP Pool",
            "name": "sfo-w01-az2-host-ip-pool01",
            "subnets": [
             {
              "cidr": "172.16.44.0/24",
              "gateway": "172.16.44.253",
              "ipAddressPoolRanges": [
               {
                "end": "172.16.44.200",
                "start": "172.16.44.10"
               }
              ]
             }
            ]
           }
          ],
          "uplinkProfiles": [
           {
            "name": "sfo-w01-az2-host-uplink-profile01",
            "teamings": [
             {
              "activeUplinks": [
               "uplink-1",
               "uplink-2"
              ],
              "name": "DEFAULT",
              "policy": "LOADBALANCE_SRCID",
              "standByUplinks": []
             }
            ],
            "transportVlan": 1644
           }
          ]
         }
        },
        "witnessSpec": {
         "fqdn": "sfo-w01-cl01-vsw01.sfo.rainpole.io",
         "vsanCidr": "172.17.11.0/24",
         "vsanIp": "172.17.11.219"
        },
        "witnessTrafficSharedWithVsanTraffic": false
       }
      }

      Note: Replace the example values in the JSON file with the correct values for your environment.
   5. Click Execute.
   6. In the confirmation dialog box, click Continue.
   7. In the Response section, expand the result section and verify that the response is SUCCEEDED.
7. Stretch the cluster with the JSON specification.
   1. Expand the APIs for managing clusters section and expand PATCH /v1/clusters/{id}.
   2. Paste the unique ID of the management cluster in the Value text-box.
   3. In the clusterUpdateSpec text box, paste the JSON specification.
      For example:

      {
       "clusterStretchSpec": {
        "hostSpecs": [
         {
          "hostname": "sfo02-w01-esx01.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 1 ID>",
          "licenseKey": "<license key>"
         },
         {
          "hostname": "sfo02-w01-esx02.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 2 ID>",
          "licenseKey": "<license key>"
         },
         {
          "hostname": "sfo02-w01-esx03.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 3 ID>",
          "licenseKey": "<license key>"
         },
         {
          "hostname": "sfo02-w01-esx04.sfo.rainpole.io",
          "hostNetworkSpec": {
           "networkProfileName": "sfo-w01-az2-nsx-np01",
           "vmNics": [
            {
             "id": "vmnic0",
             "uplink": "uplink1",
             "vdsName": "sfo-w01-cl01-vds01"
            },
            {
             "id": "vmnic1",
             "uplink": "uplink2",
             "vdsName": "sfo-w01-cl01-vds01"
            }
           ]
          },
          "id": "<ESXi host 4 ID>",
          "licenseKey": "<license key>"
         }
        ],
        "isEdgeClusterConfiguredForMultiAZ": <true, if the cluster hosts an NSX Edge cluster; false, if the cluster does not host an NSX Edge cluster>,
        "networkSpec": {
         "networkProfiles": [
          {
           "isDefault": false,
           "name": "sfo-w01-az2-nsx-np01",
           "nsxtHostSwitchConfigs": [
            {
             "ipAddressPoolName": "sfo-w01-az2-host-ip-pool01",
             "uplinkProfileName": "sfo-w01-az2-host-uplink-profile01",
             "vdsName": "sfo-w01-cl01-vds01",
             "vdsUplinkToNsxUplink": [
              {
               "nsxUplinkName": "uplink-1",
               "vdsUplinkName": "uplink1"
              },
              {
               "nsxUplinkName": "uplink-2",
               "vdsUplinkName": "uplink2"
              }
             ]
            }
           ]
          }
         ],
         "nsxClusterSpec": {
          "ipAddressPoolsSpec": [
           {
            "description": "WLD01 AZ2 Host TEP Pool",
            "name": "sfo-w01-az2-host-ip-pool01",
            "subnets": [
             {
              "cidr": "172.16.44.0/24",
              "gateway": "172.16.44.253",
              "ipAddressPoolRanges": [
               {
                "end": "172.16.44.200",
                "start": "172.16.44.10"
               }
              ]
             }
            ]
           }
          ],
          "uplinkProfiles": [
           {
            "name": "sfo-w01-az2-host-uplink-profile01",
            "teamings": [
             {
              "activeUplinks": [
               "uplink-1",
               "uplink-2"
              ],
              "name": "DEFAULT",
              "policy": "LOADBALANCE_SRCID",
              "standByUplinks": []
             }
            ],
            "transportVlan": 1644
           }
          ]
         }
        },
        "witnessSpec": {
         "fqdn": "sfo-w01-cl01-vsw01.sfo.rainpole.io",
         "vsanCidr": "172.17.11.0/24",
         "vsanIp": "172.17.11.219"
        },
        "witnessTrafficSharedWithVsanTraffic": false
       }
      }

      Note: Replace the example values in the JSON file with the correct values for your environment.
   4. Click Execute.
   5. On the confirmation dialog box, click Continue.

What to do next

Configure NSX for availability zone 2.