When determining the vSAN deployment specification for a VMware Cloud Foundation environment, you decide on the datastore size, the number of ESXi hosts per cluster, the number of disk groups per ESXi host, and the vSAN policy.

Sizing vSAN Storage

You size your vSAN datastore according to the requirements of your organization. You size the datastore for a certain node configuration as according to your hardware specifications.

Note:

Although vSAN uses thin provisioning for virtual disks, to conserve capacity consumption and prevent sudden exhaustion of the vSAN capacity in the management domain, consider the full disk space in the calculations, that is, the space for thick provisioning.

Disk space usage distribution consists of the following components:

Component

Description

Effective Raw Capacity

Space available for the vSAN datastore

Slack Space

Space reserved for vSAN specific operations such as resync and rebuilds

Dedupe Overhead

Space reserved for deduplication and compression metadata such as hash, translation, and allocations maps

Disk Formatting Overhead

Reservation for file system metadata

Checksum Overhead

Space used for checksum information

Physical Reservation

Physical space or raw capacity consumed by the overheads

Sizing the vSAN Datastore for an Environment with a Single VMware Cloud Foundation Instance

When sizing your vSAN datastore, consider the required storage capacity for all management components running in the management domain for both the management domain and for the VI workload domains you plan to deploy. Consider also the additional capacity needed for file system and any third-party management components you want to add.
Table 1. Hardware Resource Specifications of the Management Components in a Single VMware Cloud Foundation Instance

Category

Quantity

Resource Type

Consumption (GB)

Physical infrastructure (ESXi)

4

Memory

512

Management domain vCenter Server

1

Disk

694

Swap

19

SDDC Manager

1

Disk

816

Swap

16

Management domain NSX Manager Node

3

Disk

300

Swap

24

Management domain NSX Edge Node

2

Disk

200

Swap

8

VI workload domain vCenter Server 

1

Disk

920

Swap

28

VI workload domain NSX Manager Node

3

Disk

300

Swap

48

Total

  • 11 management virtual machines

  • 4 ESXi hosts

Disk

4630

Swap

295

Memory

512

Table 2. Design Decisions on the vSAN Datastore for a Single VMware Cloud Foundation Instance

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-004

Provide the default management cluster with a minimum of 13.72 TB of raw capacity for vSAN.

The management virtual machines require at least 4.4 TB of raw storage (before setting FTT to 1) and 8.8 TB when using the default vSAN storage policy.

By allocating at least 13.72 TB, initially 30% of the space is reserved for vSAN internal operations and 20% of the space is free which you can use it for additional growth of management virtual machines.

If you scale the environment out with more workloads, additional storage is required in the management domain.

VCF-MGMT-VSAN-CFG-005

On the vSAN datastore, ensure that at least 30% of free space is always available.

When vSAN reaches 80% usage, a rebalance task is started which can be resource-intensive.

Increases the amount of available storage needed.

Sizing the vSAN Datastore for an Environment with Multiple VMware Cloud Foundation Instances

In addition to the appliances you have deployed for a single VMware Cloud Foundation instance, accommodate the 3-node NSX Global Manager clusters that enable NSX Federation in the management domain and in the VI workload domains in each instance. See Logical Design for NSX-T Data Center for the Management Domain and Deployment Model for NSX Global Manager for the Management Domain.

Table 3. Hardware Resource Specifications of the Management Components in a Dual-Instance Environment

Category

Quantity

Resource Type

Consumption (GB)

ESXi

4

Memory

1024

Management domain vCenter Server

1

Disk

694

Swap

19

SDDC Manager

1

Disk

816

Swap

16

NSX Local Manager node for the management domain

3

Disk

300

Swap

24

NSX Global Manager node for the management domain

3

Disk

300

Swap

24

NSX Edge node for the management domain

2

Disk

200

Swap

8

VI workload domain vCenter Server 

1

Disk

920

Swap

28

NSX Manager node for a VI workload domain

3

Disk

300

Swap

48

NSX Global Manager node for a VI workload domain

3

Disk

300

Swap

48

Total

  • 17 management virtual machines

  • 4 ESXi hosts

Disk

6430

Swap

511

Memory

1024

Table 4. Design Decisions on the vSAN Datastore for Multiple VMware Cloud Foundation Instances

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-006

Provide the default cluster in the management with a minimum of 19.86 TB of raw capacity for vSAN.

The management virtual machines require at least 6.36 TB of raw storage (before setting FTT to 1) and 12.73 TB when using the default vSAN storage policy.

By allocating at least 19.86 TB, initially 30% of the space is reserved for vSAN internal operations and 20% of the space is free which you can use for additional growth of management virtual machines.

NFS is used as secondary shared storage for some management components, for example, for backups and log archives.

If you scale the environment out with more workloads, additional storage is required in the management domain.

Number of vSAN-Enabled Hosts per Cluster

The number of ESXi hosts in the cluster depends on these factors:

  • The amount of available space on the vSAN datastore

  • The number of failures you can tolerate in the cluster

For example, if the vSAN cluster has 3 ESXi hosts, only a single failure is supported. For a higher level of availability, you must add more hosts.

Table 5. Design Decision on the vSAN Cluster Size in a Management Domain with a Single Availability Zone

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-007

The default management cluster requires a minimum of 4 ESXi hosts to support vSAN.

  • Having 4 ESXi hosts addresses the availability and sizing requirements.

  • You can take an ESXi host offline for maintenance or upgrades without impacting the overall vSAN cluster health.

The availability requirements for the management cluster might cause underutilization of the cluster's ESXi hosts.

Table 6. Design Decision on the vSAN Cluster Size in a Management Domain with Multiple Availability Zones

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-008

To support a vSAN stretched cluster, the default management cluster requires a minimum of 8 ESXi hosts (4 in each availability zone) .

  • Having 8 ESXi hosts addresses the availability and sizing requirements.

  • You can take an availability zone offline for maintenance or upgrades without impacting the overall vSAN cluster health.

The capacity of the additional 4 hosts is not added to capacity of the cluster. They are only used to provide additional availability.

Number of vSAN Disk Groups per ESXi Host

Disk group sizing is important during volume design. The number of disk groups can affect availability and performance. The more ESXi hosts are available in the cluster, the more failures are tolerated in the cluster.

In vSAN, each ESXi host can have up to five disk groups, each containing seven capacity devices, resulting in a maximum of 35 capacity devices. The optimal number of disk groups is a balance between hardware and space requirements for the vSAN datastore.

Consider these data points when deciding on the number of disk groups per ESXi host:

  • The amount of available space on the vSAN datastore

  • The number of failures you can tolerate in the cluster

  • The performance required when recovering vSAN objects

This capability adds cost because you must provide additional hardware for the disk groups. More available disk groups can decrease the recoverability time of vSAN during a failure too.

Table 7. Design Decisions on the vSAN Disk Groups per ESXi Host

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-009

Configure vSAN with a minimum of two disk groups per ESXi host.

Reduces the size of the fault domain and spreads the I/O load over more disks for better performance.

Multiple disks groups require more disks in each ESXi host.

Sizing vSAN Disks per ESXi Host for an Environment with a Single VMware Cloud Foundation Instance

You size the vSAN disks according to the requirements for the datastore, the number of ESXi hosts in the vSAN cluster, and the number of disk groups per host.

For a single VMware Cloud Foundation instance, the required vSAN datastorse size is 37 TB of raw data. For sizing the vSAN caching tier, see the Considerations for Flash Caching Devices in vSAN post on VMware Blogs. For vSAN memory consumption by management ESXi hosts, see VMware Knowledge Base article 2113954.
Table 8. Design Decisions on the vSAN Disk Configuration for a Management Domain for a Single VMware Cloud Foundation Instance

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-010

For the caching tier in each disk group, use a flash-based drive that is at least 600 GB large.

Provides enough cache for both hybrid or all-flash vSAN configurations to buffer I/O and ensure disk group performance.

Additional space in the cache tier does not increase performance.

Larger flash disks can increase initial host cost

VCF-MGMT-VSAN-CFG-011

Allocate at least 2.3 TB of flash-based drives for the capacity tier in each disk group.

Provides enough capacity for the management virtual machines with a minimum of 30% of overhead and 20% growth when the number of primary failures to tolerate is 1.

None.

Sizing vSAN Disks per ESXi Host for an Environment with Multiple VMware Cloud Foundation Instances

For multiple VMware Cloud Foundation instances in your environment, the required vSAN datastorse size is 43 TB of raw data.

Table 9. Design Decisions on the vSAN Disk Configuration for a Management Domain for Multiple VMware Cloud Foundation Instances

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-012

Have at least 3.31 TB of flash-based drives for the capacity tier in each disk group.

Provides enough capacity for the management virtual machines with a minimum of 30% of overhead and 20% growth when the number of primary failures to tolerate is 1.

None.