Deploying a VI workload domain with a compute cluster that spans multiple racks requires Layer 3 networking and vSAN storage. To deploy such a workload domain, you use the VMware Cloud Foundation API and continue the setup after the deployment is complete.

A VI workload domain with a multi-rack compute cluster provides scalability, resilience, and optimal resource usage for customer workloads. The compute clusters in such workload domains have the following configuration:

  • Layer 3 network fabric between the racks.
  • A static IP pool with host overlay TEPs with a dedicated NSX host sub-transport node profile for each rack.
  • Each rack is a fault domain in the vSAN cluster.

For more information on the a multi-rack compute cluster design, see vSphere Cluster Design Pattern One: Multi-Rack Compute Cluster with Layer 3 Network in the VMware Cloud Foundation Design Guide.

Procedure

  1. Prepare a deployment specification for the VI workload domain in JSON format according to the requirements of your environment .
    For an example JSON specification, see VMware Cloud Foundation Planning and Preparation Workbook.
  2. In the navigation pane of the SDDC Manager UI, click Developer Center API Explorer.
  3. Expand the Domains API category.
  4. To validate the JSON specification, paste the JSON specification in the body of the POST /v1/domains/validations API and click Execute.
  5. To create the workload domain, paste the JSON specification in the body of the POST /v1/domains/ API and click Execute.
  6. Click View Task Status to view the domain creation tasks and sub tasks.

What to do next

Consider deploying two additional vSphere clusters to host the NSX Edge nodes. See NSX Edge Cluster Design Pattern Two: Multi-Rack Edge Availability in the VMware Cloud Foundation Design Guide.

Deactivate Auto Policy Management for vSAN ESA

You cannot use auto policy management in a multi-rack VI workload domain with Layer 3 network because it is not supported with vSAN ESA and fault domains.

Procedure

  1. In a Web browser, log in to the workload domain vCenter Server at https://<vcenter_server_fqdn>/ui as [email protected].
  2. In the Hosts and clusters inventory, navigate to the multi-rack vSAN cluster.
  3. On the Configure tab for the cluster, select vSAN > Services.
  4. In the vSAN ESA pane, click Edit.
  5. Toggle Auto-policy Management off and click Apply.

Create and Assign a New Default Storage Policy for vSAN ESA

By default, the vSAN ESA default storage policy is based on the number of hosts in the cluster during creation. To provide for fault domains that allow rack resiliency in a multi-rack cluster, you must create a default storage policy that is compatible with the number of racks.

Procedure

  1. In the vSphere Client for the workload domain vCenter Server, from the vSphere Client menu, select Policies and Profiles.
  2. On the VM Storage Policies page, click Create.
  3. On the Name and description page, select the vCenter Server instance that manages the multi-rack cluster.
  4. Type a name and a description for the storage policy and click Next.
  5. On the Policy structure page, select Enable rules for "vSAN" storage, and click Next.
  6. On the Availability tab of the vSAN page, set Failures to tolerate according to the configuration of your environment.
    You use RAID 5 or RAID 6 (erasure coding) for vSAN ESA.
  7. On the Storage Rules tab, select All flash and click Next.
  8. On the Storage compatibility page, verify that the datastore is compatible and click Next.
  9. On the Review and finish page, review the policy settings and click Finish.
  10. In the Storage inventory of the vSphere Client, navigate to the vSAN datastore.
  11. On the Configure tab, click General and next to the Default Storage Policy pane, click Edit.
  12. Select the storage policy that you want to assign as default to the vSAN datastore and click OK.

Create vSAN Fault Domains

Create a fault domain for each rack in the multi-rack cluster for rack resilience.

Procedure

  1. In the vSphere Client for the workload domain vCenter Server, in the Hosts and clusters inventory, navigate to the multi-rack cluster.
  2. On the Configure tab, select vSAN > Fault Domains.
  3. Click the plus icon.
  4. In the New Fault Domain dialog box, enter the fault domain name, and select the hosts for the fault domain.
  5. Click Create.
  6. Repeat Step 3 to Step 5 for the other racks.

What to do next

During a future cluster expansion, add hosts manually to their respective fault domains.