Determine the type of the capacity and caching devices, and the storage controllers for performance and stability according to the requirements of the workloads in the cluster of VI workload domain in VMware Cloud Foundation.

vSAN Physical Requirements and Dependencies

The software-defined storage module has the following requirements and options.

Requirement Category

Requirements

Number of hosts

Minimum of three ESXi hosts providing storage resources to the vSAN cluster. Recommended to use four hosts to provide full redundancy during maintenance operations.

vSAN configuration

vSAN is configured as hybrid storage or all-flash storage.

  • A vSAN hybrid storage configuration requires both magnetic devices and flash caching devices.

  • An all-flash vSAN configuration requires flash devices for both the caching and capacity tiers.

Requirements for individual hosts that provide storage resources

  • Minimum of one flash device. You must size the flash-based cache tier so that it can handle the anticipated performance requirements. For sizing the vSAN caching tier, see the Designing vSAN Disk groups – All Flash Cache Ratio Update post on VMware Blogs.

  • Minimum of two additional devices for capacity tier.

  • RAID controller that is compatible with vSAN.

  • Minimum 10 Gbps network for vSAN traffic.

  • vSphere High Availability host isolation response set to power off virtual machines. With this setting, you prevent split-brain conditions if isolation or network partition occurs. In a split-brain condition, the virtual machine might be powered on by two ESXi hosts by accident.

    See vSphere Availability Design for a Virtual Infrastructure Workload Domain.

vSAN Hardware Considerations

While VMware supports building your own vSAN cluster from compatible components, use vSAN ReadyNodes with VMware Cloud Foundation for best compatibility.

vSAN Hardware Option Description

Build Your Own

Use hardware from the VMware Compatibility Guide for the following vSAN components:

  • Flash-based drives

  • Magnetic hard drives

  • I/O controllers, including vSAN certified driver and firmware combinations

Use VMware vSAN ReadyNodes

A vSAN ReadyNode is a server configuration that is validated in a tested, certified hardware form factor for vSAN deployment, jointly recommended by the server OEM and VMware. See the vSAN ReadyNode documentation. The vSAN Compatibility Guide for vSAN ReadyNodes documentation provides examples of standardized configurations, including supported numbers of VMs and estimated number of 4K IOPS delivered.

I/O Controllers for vSAN

The I/O controllers are as important to a vSAN configuration as the selection of disk drives. vSAN supports SAS, SATA, and SCSI adapters in either pass-through or RAID 0 mode. vSAN supports multiple controllers per ESXi host. You select between single- and multi-controller configuration in the following way:

  • Multiple controllers can improve performance and mitigate a controller or SSD failure to a smaller number of drives or vSAN disk groups.

  • With a single controller, all disks are controlled by one device. A controller failure impacts all storage, including the boot media if configured.

Controller queue depth is possibly the most important aspect for performance. All I/O controllers in the VMware vSAN Hardware Compatibility Guide have a minimum queue depth of 256. Consider regular day-to-day operations and increase of I/O because of virtual machine deployment operations, or re-sync I/O activity as a result of automatic or manual fault remediation.

Table 1. Design Decisions on the Storage I/O Controller Configuration for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

VCF-WLD-vSAN-CFG-001

Ensure that the storage I/O controller that is running the vSAN disk groups is capable and has a minimum queue depth of 256 set.

Storage controllers with lower queue depths can cause performance and stability problems when running vSAN.

vSAN ReadyNode servers are configured with the right queue depths for vSAN.

Limits the number of compatible I/O controllers that can be used for storage.

vSAN Flash Options

vSAN has two configuration options: all-flash and hybrid.

Hybrid configuration

In a hybrid storage architecture, vSAN pools server-attached capacity devices (in this case magnetic devices) and flash-based caching devices, typically SSDs, or PCI-e devices, to create a distributed shared datastore.

All-flash configuration

All-flash storage uses flash-based devices (SSD or PCI-e) as a write cache while other flash-based devices provide high endurance for capacity and data persistence.

Table 2. Design Decisions on the vSAN Configuration

Decision ID

Design Decision

Design Justification

Design Implication

VCF-WLD-vSAN-CFG-002

Configure vSAN in all-flash configuration in the VI workload domain cluster.
  • Provides support for vSAN deduplication and compression.

  • Meets the performance needs of the cluster.

Using high speed magnetic disks in a hybrid vSAN configuration can provide satisfactory performance and is supported.

All vSAN disks must be flash disks, which might cost more than magnetic disks.​