This section provides guidance regarding storage considerations in VMware Cloud on AWS.
Virtual Disk Modes
VMware Cloud on AWS supports three virtual disk modes: Independent persistent, Independent nonpersistent, and Dependent.
An independent disk does not participate in virtual machine snapshots. That is, the disk state will be independent of the snapshot state; creating, consolidating, or reverting to snapshots will have no effect on the disk.
These modes have the following characteristics:
- Independent persistent
-
In this mode changes are persistently written to the disk, providing the best performance.
- Independent nonpersistent
-
In this mode disk writes are appended to a redo log. The redo log is erased when you power off the virtual machine or revert to a snapshot, causing any changes made to the disk to be discarded. When a virtual machine reads from an independent nonpersistent mode disk, VMware Cloud on AWS first checks the redo log (by looking at a directory of disk blocks contained in the redo log) and, if the relevant blocks are listed, reads that information. Otherwise, the read goes to the base disk for the virtual machine. Because of these redo logs, which track the changes in a virtual machine’s file system and allow you to commit changes or revert to a prior point in time, performance might not be as high as independent persistent mode disks.
- Dependent
-
In this mode, if a snapshot has been taken, disk writes are appended to a redo log that persists between power cycles. Thus, like the independent nonpersistent mode disks described above, dependent mode disk performance might not be as high as independent persistent mode disks. If a snapshot has not been taken, however, dependent disks are just as fast as independent disks.
Storage I/O Resource Allocation
The maximum storage I/O resources available to an object, such as a VMDK, can be set using Limits. These limits, set in I/O operations per second (IOPS), can be used to provide strict isolation and control of certain workloads. By default, these are set to unlimited; when set to any other value, VMware Cloud on AWS enforces the limits even if the underlying datastores are not fully utilized.
Limits are a cap on the object’s resource usage, not a guarantee that the object will receive that amount. Limiting some workloads can allow other, more critical workloads to maintain their performance even during peak load periods when there is contention for I/O resources.
To set Limits, from the vSphere Client right-click a virtual machine, select Edit Settings, choose the Virtual Hardware tab, expand Hard disk 1 (or whichever hard disk you want to configure), then set the Limit - IOPs to the desired value.
Running Storage Latency Sensitive Applications
By default, the VMware Cloud on AWS storage stack is configured to drive high storage throughput at low CPU cost. While this default configuration provides better scalability and higher consolidation ratios, it comes at the cost of potentially higher storage latency. Workloads that are highly sensitive to storage latency might therefore benefit from the following:
If you are using an LSILogic vHBA or a PVSCSI vHBA (with hardware version 11 or later) in the virtual machine, you can adjust the reqCallThreshold value.
The lower the reqCallThreshold value, the less time the I/O requests are likely to stay in the vHBA's queue. For instance, if reqCallThreshold is set to 1, it means when there is even one I/O request in the vHBA's queue, the I/Os will be dispatched to the lower layer. (The default reqCallThreshold value is 8.)
Reducing the reqCallThreshold value will typically increase CPU usage. It also, perhaps counter-intuitively, has the potential to reduce the maximum possible throughput on the virtual disk(s).
The system-wide configuration can be overridden for a specific vHBA. To change this variable through the vSphere Client:
Right-click the virtual machine with the vHBA you wish to change, then select Edit Settings.
Under the VM Options tab, expand Advanced, then click EDIT CONFIGURATION.
Look for scsiX.reqCallThreshold (where X is the target SCSI number) and set it to the desired reqCallThreshold value. If the variable isn’t present, click ADD CONFIGURATION PARAMS and enter it as a new parameter.
Click OK to close the Configuration Parameters window.
Click OK to close the Edit Settings window.
The change will not take effect until the virtual machine has been restarted or the relevant vHBA is disconnected from the virtual machine and connected again.
For further information on running storage latency sensitive workloads, the white paper Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere VMs, while addressing primarily network latency-sensitive workloads, might prove helpful.
