This design uses VMware vSAN to implement software-defined storage as the primary storage type.

vSAN is a hyper-converged storage solution that is integrated with the ESXi hypervisor. vSAN creates disk groups consisting of hard disk drives and flash devices or all-flash devices in the local ESXi host, and presents a highly resilient shared storage datastore to the vSphere Cluster. By using vSAN storage policies, you can control capacity, performance, and availability on a per virtual disk basis.

Table 1. vSAN Requirements

Category

Requirements

Number of ESXi hosts

Minimum of 3 ESXi hosts providing storage resources to the cluster. This can be 3 ESXi hosts or 2 ESXi hosts and 1 vSAN witness.

vSAN configuration

vSAN can be configured in all-flash or hybrid mode.

  • All-flash vSAN configuration requires flash devices for both the caching and capacity tiers.

  • vSAN hybrid storage configuration requires both the magnetic devices for capacity and the flash devices for caching.

Requirements for individual ESXi hosts that provide storage resources.

  • Minimum of one flash device. The flash cache tier must be at least 10% of the size of the capacity tier.

  • Minimum of two HDDs for hybrid mode, or an additional flash device for an all-flash configuration.

  • RAID controller that is compatible with vSAN with a separate controller for the boot device.

  • Minimum 10 Gbps network for vSAN traffic.

  • Host isolation response of vSphere High Availability is set to power off VMs.

I/O Controllers

The I/O controllers are as important as the selection of disk drives to a vSAN configuration. vSAN supports SAS, SATA, and SCSI adapters in either the pass-through or RAID 0 mode. vSAN supports multiple controllers per ESXi host.

  • Multi-Controller Configuration: Multiple controllers can improve performance and mitigate a controller or SSD failure to a smaller number of drives or vSAN disk groups.

  • Single-Controller Configuration: With a single controller, all disks are controlled by one device. The failure of a controller impacts all storage, including the boot media (if configured).

Controller queue depth is an important aspect of performance. All I/O controllers in the VMware vSAN Hardware Compatibility Guide have a minimum queue depth of 256. If you increase the queue depth to a value higher than 256, ensure that you consider the regular day-to-day operations in your environment. Examples of events that require higher queue depth are as follows:

  • VM deployment operations

  • Re-sync I/O activity as a result of automatic or manual fault remediation.

Table 2. Recommended Physical Storage Design

Design Recommendation

Design Justification

Design Implication

Use all-flash vSAN in all clusters.

  • Provides best performance with low latency.

  • When using all-flash vSAN, you can enable de-duplication and compression that saves space on the datastores.

Flash storage may cost more than traditional magnetic disks.

For the management cluster, provide a vSAN configuration with at least 6 TB of usable space.

Provides all the required space for this solution while allowing the deployment of additional monitoring and management components in the management cluster.

On day 1, more space is required.

For the edge cluster, provide a vSAN configuration with at least 500 GB of usable space.

Provides required storage to run NSX Edge Nodes.

None

For the compute clusters, size the vSAN datastore according to the current workloads plus 5 years of expected growth.

Ensures that the storage solution is not required to be upgraded that can cause downtime to workloads.

On day 1, more space is required.
Note:

This design uses vSAN. Any supported storage solution can be used as long as it meets the characteristics of this storage design. For best practices, see the vendor documentation.