Determine the type of the capacity and caching devices, and the storage controllers for performance and stability according to the requirements of the management components of VMware Cloud Foundation.

vSAN Physical Requirements and Dependencies

vSAN has the following requirements and options.

Table 1. vSAN Physical Requirements

Requirement Category

Requirements

Number of hosts

Minimum of four ESXi hosts providing storage resources to the vSAN cluster in the management domain.

vSAN configuration

vSAN is configured as hybrid storage or all-flash storage.

  • A vSAN hybrid storage configuration requires both magnetic devices and flash caching devices.

  • An all-flash vSAN configuration requires flash devices for both the caching and capacity tiers.

Requirements for individual hosts that provide storage resources

  • Minimum of one flash device. The flash-based cache tier must be at least 10% of the size of the HDD capacity tier.

  • Minimum of two additional devices for capacity tier.

  • RAID controller that is compatible with vSAN.

  • Minimum 10 Gbps network for vSAN traffic.

  • vSphere HA host isolation address that is set to the vSAN network gateway address or another valid IP address in the same network.

  • vSphere High Availability host isolation response set to power off virtual machines. With this setting, you prevent split-brain conditions if isolation or network partition occurs. In a split-brain condition, the virtual machine might be powered on by two ESXi hosts by accident.

    See vSphere Availability Design for the Management Domain.

vSAN Hardware Considerations

While VMware supports building your own vSAN cluster from compatible components, use vSAN ReadyNodes with VMware Cloud Foundation for best compatibility. See Design Decisions on the ESXi Server Hardware.

Table 2. vSAN Hardware Considerations

vSAN Hardware Options

Description

Build Your Own

Use hardware from the VMware Compatibility Guide for the following vSAN components:

  • Flash-based drives

  • Magnetic hard drives

  • I/O controllers, including vSAN certified driver and firmware combinations

Use VMware vSAN ReadyNodes

A vSAN ReadyNode is a server configuration that is validated in a tested, certified hardware form factor for vSAN deployment, jointly recommended by the server OEM and VMware. See the vSAN ReadyNode documentation. The vSAN Compatibility Guide for vSAN ReadyNodes documentation provides examples of standardized configurations, including supported numbers of VMs and estimated number of 4K IOPS delivered.

I/O Controllers for vSAN

The I/O controllers are as important to a vSAN configuration as the selection of disk drives. vSAN supports SAS, SATA, and SCSI adapters in either pass-through or RAID 0 mode. vSAN supports multiple controllers per ESXi host. You select between single- and multi-controller configuration in the following way:

  • Multiple controllers can improve performance, and mitigate a controller or SSD failure to a smaller number of drives or vSAN disk groups.

  • With a single controller, all disks are controlled by one device. A controller failure impacts all storage, including the boot media if configured.

Controller queue depth is possibly the most important aspect for performance. All I/O controllers in the VMware vSAN Hardware Compatibility Guide have a minimum queue depth of 256. Consider regular day-to-day operations and increase of I/O because of virtual machine deployment operations, or re-sync I/O activity as a result of automatic or manual fault remediation.

Table 3. Design Decisions on Storage I/O Controller Configuration for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-001

Ensure that the storage I/O controller that is running the vSAN disk groups is capable and has a minimum queue depth of 256 set.

Storage controllers with lower queue depths can cause performance and stability problems when running vSAN.

vSAN ReadyNode servers are configured with the right queue depths for vSAN.

Limits the number of compatible I/O controllers that can be used for storage.

VCF-MGMT-VSAN-CFG-002

Do not use the storage I/O controllers that are running vSAN disk groups for another purpose.

Running non-vSAN disks, for example, VMFS, on a storage I/O controller that is running a vSAN disk group can impact vSAN performance.

If non-vSAN disks are required in ESXi hosts, you must have an additional storage I/O controller in the host.

vSAN Flash Options

vSAN has two configuration options: all-flash and hybrid. To provide good performance of the management components, use all-flash vSAN configuration for the management domain in VMware Cloud Foundation.

Hybrid configuration

In a hybrid storage architecture, vSAN pools server-attached capacity devices (in this case magnetic devices) and flash-based caching devices, typically SSDs, or PCI-e devices, to create a distributed shared datastore.

All-flash configuration

All-flash storage uses flash-based devices (SSD or PCI-e) as a write cache while other flash-based devices provide high endurance for capacity and data persistence.

Table 4. Design Decisions on vSAN Configuration

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-CFG-003

Configure vSAN in all-flash configuration in the default management cluster.

Meets the performance needs of the default management cluster.

Using high-speed magnetic disks in a hybrid vSAN configuration can provide satisfactory performance and is supported.

All vSAN disks must be flash disks, which might cost more than magnetic disks.