Tenant workloads run on the ESXi hosts in the shared edge and compute cluster. Because of the shared nature of the cluster, NSX Controllers and Edge devices run in this cluster. This cluster design determines the number of ESXi hosts, vSphere HA settings, and several other characteristics of the shared edge and compute cluster.

Table 1. Design Decisions on the Shared Edge and Compute Cluster

Decision ID

Design Decision

Design Justification

Design Implication

SDDC-VI-VC-022

Create a shared edge and compute cluster for the NSX Controller nodes and NSX Edge gateway devices.

NSX Manager requires a one-to-one relationship with a vCenter Server system.

  • Each time you provision a Compute vCenter Server system, a new NSX Manager is required.

  • You set anti-affinity rules to keep each controller on a separate ESXi host.

    A 4-node cluster supports maintenance while ensuring that the 3 controllers remain on separate ESXi hosts.

SDDC-VI-VC-023

When using a single availability zone, configure Admission Control for 1 ESXi host failure and percentage-based failover capacity.

vSphere HA protects the NSX Controller instances and edge services gateway devices in the event of an ESXi host failure. vSphere HA powers on virtual machines from the failed ESXi hosts on any remaining ESXi hosts.

Only a single ESXi host failure is tolerated before potential resource contention.

SDDC-VI-VC-024

When using two availability zones, configure Admission Control for percentage-based failover based on half of the ESXi hosts in the cluster.

For example, in a cluster with 8 ESXi hosts you configure admission control for 4 ESXi hosts failure and percentage-based failover capacity.

vSphere HA protects the NSX Controller instances and edge services gateway devices in the event of an ESXi host failure. vSphere HA powers on virtual machines from the failed ESXi hosts on any remaining ESXi hosts.

Only half of a stretched cluster should be used to ensure that all VMs have enough resources in an availability zone outage.

You must add ESXi hosts to the cluster in pairs, one in each availability zone.

SDDC-VI-VC-025

In Region A, create a shared edge and compute cluster with a minimum of 4 ESXi hosts for a single availability zone or with a minimum of 8 ESXi hosts for two availability zones (minimum of 4 ESXi hosts in each availability zone).

Allocating 4 ESXi hosts provides full redundancy for each availability zone within the cluster.

Having 4 ESXi hosts in each availability zone guarantees vSAN and NSX redundancy during availability zone outages or maintenance operations.

4 ESXi hosts is the smallest starting point for a single availability zone and 8 ESXi hosts for two availability zones for the shared edge and compute cluster for redundancy and performance thus increasing cost.

SDDC-VI-VC-026

In Region B, create a shared edge and compute cluster with a minimum of 4 hosts.

  • 3 NSX Controller nodes are required for sufficient redundancy and majority decisions.

  • 1 ESXi host is available for failover and to allow for scheduled maintenance.

4 ESXi hosts is the smallest starting point for the shared edge and compute cluster for redundancy and performance thus increasing cost over a 3-node cluster.

SDDC-VI-VC-027

Set up VLAN-backed port groups for external access and management on the shared edge and compute cluster ESXi hosts.

Edge gateways need access to the external network in addition to the management network.

VLAN-backed port groups must be configured with the correct number of ports, or with elastic port allocation.

SDDC-VI-VC-028

Create a resource pool for the required SDDC NSX Controller nodes and edge appliances with a CPU share level of High, a memory share of Normal, and 16 GB memory reservation.

The NSX components control all network traffic in and out of the SDDC and update route information for inter-SDDC communication. In a contention situation, these virtual machines must receive all the resources required.

During contention, SDDC NSX components receive more resources than all other workloads. As a result, monitoring and capacity management of tenant workloads must be a proactive activity.

SDDC-VI-VC-029

Create a resource pool for all user NSX Edge devices with a CPU share value of Normal and a memory share value of Normal.

NSX edges for users, created by vRealize Automation, support functions such as load balancing for user workloads. These edge devices do not support the entire SDDC as such they receive a lower amount of resources during contention.

During contention, these NSX edges will receive fewer resources than the SDDC edge devices. As a result, monitoring and capacity management must be a proactive activity.

SDDC-VI-VC-030

Create a resource pool for all user virtual machines with a CPU share value of Normal and a memory share value of Normal.

Creating virtual machines outside of a resource pool will have a negative impact on all other virtual machines during contention. In a shared edge and compute cluster, the SDDC edge devices must be guaranteed resources before all other workloads as to retain network connectivity. Setting the share values to Normal gives the SDDC edges more shares of resources during contention ensuring network traffic is not impacted.

During contention, tenant virtual machines might require resources and experience poor performance. Proactively perform monitoring and capacity management, add capacity or dedicate an edge cluster before contention occurs.

SDDC-VI-VC-031

When using two availability zones, set the cluster isolation addresses for the cluster to the gateway IP addresses for the vSAN network in both availability zones.

vSphere HA can validate complete network isolation in the case of a connection failure between availability zones.

You must manually configure the isolation address.

SDDC-VI-VC-032

When using two availability zones, set the advanced cluster setting das.usedefaultisolationaddress to false.

Ensures that the manual isolation addresses are used instead of the default management network gateway address.

None.

SDDC-VI-VC-033

When using a single availability zone, create a host profile for the shared edge and compute cluster.

Simplifies the configuration of ESXi hosts and ensures that the settings are uniform across the cluster.

Anytime an authorized change to an ESXi host is made, you must update the host profile to reflect the change or the status will show non-compliant.

SDDC-VI-VC-034

When using two availability zones, create a host profile for each availability zone in the cluster.

Simplifies configuration of ESXi hosts and ensures that the settings are uniform across the availability zones in the cluster.

Anytime an authorized change to an ESXi host is made, you must update the host profile to reflect the change or the status will show non-compliant.

Because of configuration differences between availability zones, two host profiles are required and must be applied on each ESXi host.

The shared edge and compute cluster logical design has the following attributes. The number of VMs on the shared edge and compute cluster will start low but will grow quickly as user workloads are created.

Table 2. Shared Edge and Compute Cluster Logical Design Background
Attribute Specification
Minimum number of ESXi hosts required to support the shared edge and compute cluster 3
Number of ESXi hosts recommended because of operational constraints (ability to take an ESXi host offline without sacrificing high availability capabilities) 4
Number of ESXi hosts recommended because of operational constraints, while using vSAN (ability to take an ESXi host offline without sacrificing high availability capabilities)
  • 4 (single availability zone)
  • 8 (two availability zones)
Capacity for ESXi host failures per cluster
  • 25% reserved CPU RAM (single availability zone)
  • 50% reserved CPU RAM (two availability zones)