Set Latency Sensitive Threshold

When you use the HPP for your storage devices, set the latency sensitive threshold for the device, so that I/O can avoid the I/O scheduler.

By default, ESXi passes every I/O through the I/O scheduler. However, using the scheduler might create internal queuing, which is not efficient with the high-speed storage devices.

You can configure the latency sensitive threshold and enable the direct submission mechanism that helps I/O to bypass the scheduler. With this mechanism enabled, the I/O passes directly from PSA through the HPP to the device driver.

For the direct submission to work properly, the observed average I/O latency must be lower than the latency threshold you specify. If the I/O latency exceeds the latency threshold, the system stops the direct submission and temporarily reverts to using the I/O scheduler. The direct submission is resumed when the average I/O latency drops below the latency threshold again.

You can set the latency threshold for a family of devices claimed by HPP. Set the latency threshold using the vendor and model pair, the controller model, or PCIe vendor ID and sub vendor ID pair.

Procedure

1. Set the latency sensitive threshold for the device by running the following command:
   esxcli storage core device latencythreshold set -t value in milliseconds

   Use one of the following options.

   Option	Example
   Vendor/model	Set the latency sensitive threshold parameter for all devices with the indicated vendor and model: esxcli storage core device latencythreshold set -v 'vendor1' -m 'model1' -t 10
   NVMe controller model	Set the latency sensitive threshold for all NVMe devices with the indicated controller model: esxcli storage core device latencythreshold set -c 'controller_model1' -t 10
   PCIe vendor/subvendor ID	Set the latency sensitive threshold for devices with 0x8086 as PCIe vendor ID and 0x8086 as PCIe sub vendor ID. esxcli storage core device latencythreshold set -p '8086' -s '8086' -t 10

2. Verify that the latency threshold is set:
   esxcli storage core device latencythreshold list

   Device                Latency Sensitive Threshold
   --------------------  ---------------------------
   naa.55cd2e404c1728aa               0 milliseconds
   naa.500056b34036cdfd               0 milliseconds
   naa.55cd2e404c172bd6              50 milliseconds
   

3. Monitor the status of the latency sensitive threshold. Check VMkernel logs for the following entries:
   • Latency Sensitive Gatekeeper turned on for device device. Threshold of XX msec is larger than max completion time of YYY msec
   • Latency Sensitive Gatekeeper turned off for device device. Threshold of XX msec is exceeded by command completed in YYY msec