This design uses VMware vSAN to implement software-defined storage as the primary storage type.

vSAN is a hyper-converged storage software that is integrated with the ESXi hypervisor. vSAN creates a cluster of hard disk drives and flash devices in the local ESXi host, and presents a flash-optimized, highly resilient, shared storage datastore to ESXi hosts and VMs. By using vSAN storage policies, you can control capacity, performance, and availability per virtual disk.

Table 1. vSAN Requirements

Category

Requirements

Number of ESXi hosts

  • Minimum of 3 ESXi hosts providing storage resources to the vSAN cluster. This can be 3 ESXi hosts or 2 ESXi hosts and 1 vSAN witness.

  • Minimum of 4 ESXi hosts for vSAN automated rebuild.

vSAN configuration

vSAN can be configured in all-flash or hybrid mode:

  • All-flash mode: All-flash vSAN configuration requires flash devices for both the caching and capacity tiers.

  • Hybrid mode: vSAN hybrid storage configuration requires both magnetic devices and flash caching devices.

Individual ESXi hosts that provide storage resources.

  • Minimum of one flash device. The flash cache tier must be at least 10% of the size of the capacity tier.

  • Minimum of two HDDs for hybrid mode, or an additional flash device for an all-flash configuration.

  • RAID controller that is compatible with vSAN.

  • Minimum 10 Gbps network for vSAN traffic.

  • Host isolation response of vSphere High Availability is set to power off VMs.

I/O Controllers

The I/O controllers are as important as the selection of disk drives to a vSAN configuration. vSAN supports SAS, SATA, and SCSI adapters in either pass-through or RAID 0 mode. vSAN supports multiple controllers per ESXi host.

  • Multi-Controller Configuration: Multiple controllers can improve performance and mitigate a controller or SSD failure to a smaller number of drives or vSAN disk groups.

  • Single-Controller Configuration: With a single controller, all disks are controlled by one device. A controller failure impacts all storage, including the boot media (if configured).

Controller queue depth is an important aspect of performance. All I/O controllers in the VMware vSAN Hardware Compatibility Guide have a minimum queue depth of 256. If you increase the queue depth to a value higher than 256, ensure that you consider the regular day-to-day operations in your environment. Examples of events that require higher queue depth are as follows:

  • VM deployment operations

  • Re-sync I/O activity because of automatic or manual fault remediation

Table 2. Recommended Physical Storage Design

Design Decision

Design Justification

Design Implication

Use all-flash vSAN in all vSphere clusters.

  • Provides the best performance with low latency.

  • When using all-flash vSAN, you can enable de-duplication and compression that saves space on the datastores.

Flash storage costs more than traditional magnetic disks.

For the management cluster, provide a vSAN configuration with at least 6 TB of usable space.

Provides all the required space for this solution while allowing the deployment of additional monitoring and management components in the management cluster.

On day 1, more space is required.

For the edge cluster, provide a vSAN configuration with at least 500 GB of usable space.

Provides required storage to run NSX Edge Nodes.

None

For the compute clusters, size the vSAN datastore according to the current workloads plus five years of expected growth.

Ensures that the storage solution is not required to be upgraded as it can cause downtime to workloads.

On day 1, more space is required.
Note:

This design uses vSAN. You can use any supported storage solution that meets the characteristics of this storage design. For best practices, see the vendor documentation.