Plan the configuration of flash devices for Virtual SAN cache and all-flash capacity to provide high performance and required storage space, and to accommodate future growth.

Choosing Between PCIe or SSD Flash Devices

Choose PCIe or SSD flash devices according to the requirements for performance, capacity, write endurance, and cost of the Virtual SAN storage.

  • Compatibility. The model of the PCIe or SSD devices must be listed in the Virtual SAN section of the VMware Compatibility Guide.
  • Performance. PCIe devices generally have faster performance than SSD devices.
  • Capacity. The maximum capacity that is available for PCIe devices is generally greater than the maximum capacity that is currently listed for SSD devices for Virtual SAN in the VMware Compatibility Guide.
  • Write endurance. The write endurance of the PCIe or SSD devices must meet the requirements for capacity or for cache in all-flash configurations, and for cache in hybrid configurations.

    For information about the write endurance requirements for all-flash and hybrid configurations, see the VMware Virtual SAN Design and Sizing Guide. For information about the write endurance class of PCIe and SSD devices, see the Virtual SAN section of the VMware Compatibility Guide.

  • Cost. PCIe devices generally have higher cost than SSD devices.

Flash Devices as Virtual SAN Cache

Design the configuration of flash cache for Virtual SAN for write endurance, performance, and potential growth based on these considerations.

Table 1. Sizing Virtual SAN Cache
Storage Configuration Considerations
All-flash and hybrid configurations
  • The flash caching device must provide at least 10 percent of the anticipated storage that virtual machines are expected to consume, not including replicas such as mirrors.

    The Primary level of failures to tolerate attribute from the VM storage policy does not impact the size of the cache.

  • A higher cache-to-capacity ratio eases future capacity growth. Oversizing cache enables you to easily add more capacity to an existing disk group without the need to increase the size of the cache.
  • Flash caching devices must have high write endurance.
  • When a flash caching device is at the end of its life, replacing it is more complicated than replacing a capacity device because such an operation impacts the entire disk group.
  • If you add more flash devices to increase the size of the cache, you must create more disk groups. The ratio between flash cache devices and disk groups is always 1:1.

    A configuration of multiple disk groups provides the following advantages:

    • Reduced risk of failure because fewer capacity devices are affected if a single caching device fails
    • Potentially improved performance if you deploy multiple disk groups that contain smaller flash caching devices.

    However, when you configure multiple disk groups, the memory consumption of the hosts increases.

All-flash configurations

In all-flash configurations, Virtual SAN uses the cache layer for write caching only. The write cache must be able to handle very high write activities. This approach extends the life of capacity flash that might be less expensive and might have lower write endurance.

Hybrid configurations

If the read cache reservation is configured in the active VM storage policy for performance reasons, the hosts in the Virtual SAN cluster must have sufficient cache to satisfy the reservation during a post-failure rebuild or maintenance operation.

If the available read cache is not sufficient to satisfy the reservation, the rebuild or maintenance operation fails. Use read cache reservation only if you must meet a specific, known performance requirement for a particular workload.

The use of snapshots consumes cache resources. If you plan to use several snapshots, consider dedicating more cache than the conventional 10 percent cache-to-consumed-capacity ratio.