You size the compute resources of the ESXi hosts in the management domain according to the system requirements of the management components and the requirements for managing customer workloads hosted on VMware Cloud Foundation.

ESXi Server Hardware

The configuration and assembly process for each system should be standardized, with all components installed in the same manner on each ESXi host. Because standardization of the physical configuration of the ESXi hosts removes variability, the infrastructure is easily managed and supported. ESXi hosts are deployed with identical configuration across all cluster members, including storage and networking configurations. For example, consistent PCIe card slot placement, especially for network controllers, is essential for accurate mapping of physical network controllers to virtual network resources. By using identical configurations, you have an even balance of virtual machine storage components across storage and compute resources.

Table 1. Design Decisions on the ESXi Server Hardware

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-001

Use vSAN ReadyNodes with vSAN storage for each ESXi host in the management domain.

Your management domain is fully compatible with vSAN at deployment.

For information about the models of physical servers that are vSAN-ready, see vSAN Compatibility Guide for vSAN ReadyNodes.

Hardware choices might be limited.

If you plan to use a server configuration that is not a vSAN ReadyNode, your CPU, disks and I/O modules must be listed on the VMware Compatibility Guide under CPU Series and vSAN Compatibility List aligned to the ESXi version specified in VMware Cloud Foundation 4.5 Release Notes.

VCF-MGMT-ESX-CFG-002

Allocate hosts with uniform configuration across the default management vSphere cluster.

A balanced cluster has these advantages:

  • Predictable performance even during hardware failures

  • Minimal impact of resync or rebuild operations on performance

You must apply vendor sourcing, budgeting, and procurement considerations for uniform server nodes on a per cluster basis.
In VMware Cloud Foundation, the principal storage system for the management domain is vSAN. See Deployment Specification of vSAN for the Management Domain
Table 2. Design Decisions on the ESXi Server Hardware

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-001

Use vSAN ReadyNodes with vSAN storage for each ESXi host in the management domain.

Your management domain is fully compatible with vSAN at deployment.

For information about the models of physical servers that are vSAN-ready, see vSAN Compatibility Guide for vSAN ReadyNodes.

Hardware choices might be limited.

If you plan to use a server configuration that is not a vSAN ReadyNode, your CPU, disks and I/O modules must be listed on the VMware Compatibility Guide under CPU Series and vSAN Compatibility List aligned to the ESXi version specified in VMware Cloud Foundation 4.5 Release Notes.

VCF-MGMT-ESX-CFG-002

Allocate hosts with uniform configuration across the default management vSphere cluster.

A balanced cluster has these advantages:

  • Predictable performance even during hardware failures

  • Minimal impact of resync or rebuild operations on performance

You must apply vendor sourcing, budgeting, and procurement considerations for uniform server nodes on a per cluster basis.

ESXi Host CPU and CPU Overcommitment

When sizing CPU capacity for the ESXi hosts in the management domain, consider these parameters:

  • The requirements for the management workloads.

  • Scenarios where one host is not available because of failure or maintenance. In these cases, keep the CPU overcommitment ratio vCPU-to-pCPU less than or equal to 2:1.

  • Expected number of VI workload domains.

  • Additional third-party management components.

Size your CPU according to the number of physical cores, not the logical cores. Simultaneous multithreading (SMT) technologies in CPUs, such as hyper-threading in Intel CPUs, improve CPU performance by allowing multiple threads to run in parallel on the same CPU core. Although a single CPU core can be viewed as two logical cores, the performance enhancement will not be equivalent to 100% more CPU power. It will also differ from one environment to another.
In this design, you calculate the number of physical CPU cores on each server for a basic configuration of the management domain that consists of 4 ESXi hosts and runs the vCenter Server instances for the management and VI workload domains, NSX Manager clusters for the management and VI workload domains, NSX Edge cluster for the management domain, and SDDC Manager.
If you plan to have more ESXi hosts, or use additional VMware solutions or third-party management components, then recalculate the number of physical CPU cores.
Table 3. Design Decisions on the ESXi CPU Configuration for an Environment with Multiple VMware Cloud Foundation Instances

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-005

Install each ESXi host in the default, four-node, management cluster of each instance with a minimum of 22 physical CPU cores.

  • The management components in the cluster require a total of 132 vCPUs.

  • If one of the hosts is not available because of a failure or maintenance event, the CPU overcommitment ratio becomes 2:1.

If you plan to add more than one virtual infrastructure workload domain, additional VMware solution or third-party management components, you must add more CPU cores to the management ESXi hosts.
Table 4. Design Decisions on the ESXi CPU Configuration for an Environment with a Single VMware Cloud Foundation Instance

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-003

Install each ESXi host in the default, four-node, management vSphere cluster with a minimum of 13 physical CPU cores.

  • The management components in the cluster require a total of 78 vCPUs.

  • If one of the hosts is not available because of a failure or maintenance event, the CPU overcommitment ratio becomes 2:1.

If you plan to add more than one virtual infrastructure workload domain, additional VMware solutions or third-party management components, you must add more CPU cores to the management ESXi hosts.

VCF-MGMT-ESX-CFG-004

When sizing CPU, do not consider multithreading technology and associated performance gains.

Although multithreading technologies increase CPU performance, the performance gain depends on running workloads and differs from one case to another.

Because you must provide more physical CPU cores, costs increase and hardware choices become limited.
Table 5. Design Decisions on the ESXi CPU Configuration for an Environment with Multiple VMware Cloud Foundation Instances

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-005

Install each ESXi host in the default, four-node, management cluster of each instance with a minimum of 22 physical CPU cores.

  • The management components in the cluster require a total of 132 vCPUs.

  • If one of the hosts is not available because of a failure or maintenance event, the CPU overcommitment ratio becomes 2:1.

If you plan to add more than one virtual infrastructure workload domain, additional VMware solution or third-party management components, you must add more CPU cores to the management ESXi hosts.

ESXi Host Memory

When sizing memory for the ESXi hosts in the management domain, consider certain requirements.

  • Requirements for the management components that are running in the cluster

    When sizing memory for the hosts in a cluster, to reserve the resources of one host for failover or maintenance, set the admission control setting to N+1, which reserves the resources of one host for failover or maintenance.

  • Number of vSAN disk groups and disks on an ESXi host

    To support the maximum number of disk groups, you must provide 32 GB of RAM. For more information about disk groups, including design and sizing guidance, see Administering VMware vSAN from the vSphere documentation and Sizing vSAN Resources for the Management Domain.

In this design, you calculate the memory size of each server for a basic configuration of the management domain that consists of 4 ESXi hosts and runs the vCenter Server instances for the management and VI workload domains, NSX Manager clusters for the management and VI workload domains, NSX Edge cluster for the management domain, and SDDC Manager.
If you plan to have more ESXi hosts, or use additional VMware solutions or third-party management components, then recalculate the required memory size.
Table 6. Design Decisions on the ESXi Memory Size for an Environment with a Single VMware Cloud Foundation Instance

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-006

Install each ESXi host in the default, four-node, management cluster with a minimum of 128 GB RAM.

The management components in this cluster require a total of 295 GB RAM.

You allocate the remaining memory to additional management components that are required for new capabilities, for example, for new virtual infrastructure workload domains.

  • In a four-node cluster, only 384 GB is available for use because the host redundancy in vSphere HA is configured to N+1.

  • If you plan to add more than one virtual infrastructure workload domain, additional VMware solutions or third-party management components, you must add more memory to the management ESXi hosts.
Table 7. Design Decisions on the ESXi Memory Size for an Environment with Multiple VMware Cloud Foundation Instances

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-ESX-CFG-007

Install each ESXi host in the default, four-node, management cluster with a minimum of 256 GB RAM.

The management components in this cluster require a total of 511 GB RAM.

You allocate the remaining memory to additional management components that are required for new capabilities, for example, for new virtual infrastructure workload domains

  • In a four-node cluster, only 768 GB is available for use because the host redundancy that is configured in vSphere HA is N+1.

  • If you plan to add more than one virtual infrastructure workload domain, additional VMware solutions or third-party management components, you must add more memory to the management ESXi hosts.