VMware Cloud on AWS - Storage

VMware Cloud on AWS utilizes vSAN as its mechanism for providing distributed storage within each Cluster of the SDDC.

First vSphere Cluster in an SDDC

Within the first vSphere Cluster of the SDDC, vSAN has been modified to present two logical Datastores from the same underlying physical storage: one for management appliances and the other for end-user workloads. This separation exists purely as a means of enforcing permissions on the storage for management appliances.

vSAN Reserved Space

vSAN requires that a certain percentage of raw storage within a Datastore be reserved, also referred to as slack space. This reserved space is used for operations such as deduplication, object re-balancing, and for recovering from hardware outages within the underlying pool of storage capacity. The current official recommendation is to maintain a 20% buffer for slack space.

As part of its service level agreement with customers, VMware will ensure the health of the SDDC by enforcing this slack space requirement using storage-based EDRS scale-up whenever necessary. EDRS will automatically scale up the SDDC if available slack space drops to 20% or less.

Deduplication and Compression

Deduplication and Compression are designed to optimize storage by reducing the amount of space required to store data within the Datastore. These features are enabled on vSAN Datastores by default and cannot be disabled. Although highly dependent on the type of data being stored, a 2x reduction is a good general guideline for most deployments.

Storage Policy Based Management

Storage Policy Based Management (SPBM) is a declarative system for the management of storage within the SDDC. SPBM allows for the definition of a policy that defines specific data protection and performance criteria and permits it to be applied granularly to objects within vCenter Server. SPBM allows multiple policies to coexist and permits them to be applied at both the VM and VMDK level.

Once defined, these policies may be extended across clusters. The Policy configuration and assignment of any and all management appliances is controlled by VMware^® and may not be modified. The default policy configuration for any workload Datastore is also maintained by the service but customers may create and assign custom policies if they wish to override this default behavior.

The first consideration when designing policy is availability, which defines the level of redundancy desired for objects within the datastore. The second consideration for policy design is with performance, which defines both the processing overhead required to implement the policy and the overall end-user impact in terms of IOPS for a given workload.

Availability

Availability is broken down into 2 levels: the ability to survive a site-level disaster and the ability to survive failures internal to a given site. Within the context of VMware Cloud on AWS, a site falls within the boundary of an AWS Availability Zone (AZ). The ability of a VM or VMDK to survive an AZ-level outage applies only to stretched cluster designs. Specifically, the “dual site mirror” option for stretched clusters will cause data to be replicated across AZs and will enable an object to survive a complete failure of either AZ.

Within an AZ, there are two settings to consider for data resiliency.

• Failures to Tolerate (FTT) - This defines the number of host or disk failures to tolerate. In other words, it defines the number of devices that can fail before data loss occurs.

• Fault Tolerance Method (FTM) - This defines the type of data replication used: mirroring (RAID1) or erasure coding (RAID5/RAID6)

The options available for FTT/FTM will be determined by the minimum number of hosts within a cluster. The options chosen will impact the total amount of usable capacity of the datastore.

Performance

Data that is stored using an erasure coding FTM will consist of the actual data itself, which is broken into segments, along with parity copies. While erasure coding provides better efficiency in terms of storage overhead, this practice of segmenting data with added parity comes at a cost to performance. This performance penalty is especially evident during failure scenarios when data must be calculated from parity.

vSAN Design Decisions

Design Decision	Design Justification	Design Implication
For high I/O workloads create an SPBM policy that sets the FTM to RAID-1 (Mirroring)	Provides the highest available throughput.	Erasure coding provides better storage efficiency.
On all vSAN datastores ensure that at least 20% of free space is always available.	When a vSAN datastore reaches 80% utilization, EDRS will add capacity by adding an additional ESXi host. When vSAN reaches 80% utilization, a rebalance task is started which can be resource-intensive.	Increases the amount of available storage needed.