With hardware acceleration, your ESXi host can integrate with block storage devices, Fibre Channel, or iSCSI, and use storage array operations that include full copy, block zeroing, and hardware assisted locking.

ESXi hardware acceleration supports the following array operations:

  • Full copy, also called clone blocks or copy offload. Storage arrays can make full copies of data within the array without having the host read and write the data. This operation reduces the time and network load when cloning virtual machines, provisioning from a template, or migrating with vMotion.
  • Block zeroing, also called write same. Storage arrays can zero out a large number of blocks to provide newly allocated storage, free of previously written data. This operation reduces the time and network load when creating virtual machines and formatting virtual disks.
  • Hardware assisted locking, also called atomic test and set (ATS). Supports discrete virtual machine locking without use of SCSI reservations. This operation allows disk locking per sector, instead of the entire LUN as with SCSI reservations.

Check with your vendor for the hardware acceleration support. Certain storage arrays require that you activate the support on the storage side.

On your host, the hardware acceleration is available by default. If your storage does not support the hardware acceleration, you can deactivate it.

In addition to hardware acceleration support, ESXi includes support for array thin provisioning. For information, see ESXi and Array Thin Provisioning.

Deactivate Hardware Acceleration for Block Storage Devices

On your host, the hardware acceleration for block storage devices is activated by default. You can use the vSphere Client advanced settings to deactivate the hardware acceleration operations.

As with any advanced settings, before you deactivate the hardware acceleration, consult with the VMware support team.

Procedure

  1. In the vSphere Client, navigate to the ESXi host.
  2. Click the Configure tab.
  3. Under System, click Advanced System Settings.
  4. To deactivate the hardware acceleration, change the value for these options to 0:
    • VMFS3.HardwareAcceleratedLocking
    • DataMover.HardwareAcceleratedMove
    • DataMover.HardwareAcceleratedInit

Managing Hardware Acceleration on Block Storage Devices

To integrate with the block storage arrays, vSphere uses the ESXi extensions called Storage APIs - Array Integration (VAAI). With this integration, vSphere can use the array hardware operations.

In the vSphere 5.x and later releases, these extensions are implemented as the T10 SCSI commands. As a result, with the devices that support the T10 SCSI standard, your ESXi host can communicate directly and does not require the VAAI plug-ins.

If the device does not support T10 SCSI or provides partial support, ESXi reverts to using the VAAI plug-ins, installed on your host. The host can also use a combination of the T10 SCSI commands and plug-ins. The VAAI plug-ins are vendor-specific and can be either VMware or partner developed. To manage the VAAI capable device, your host attaches the VAAI filter and vendor-specific VAAI plug-in to the device.

For information about whether your storage requires VAAI plug-ins or supports hardware acceleration through T10 SCSI commands, see the VMware Compatibility Guide or contact your storage vendor.

You can use several esxcli commands to query storage devices for the hardware acceleration support information. For the devices that require the VAAI plug-ins, the claim rule commands are also available. For information about esxcli commands, see Getting Started with ESXCLI.

Display Hardware Acceleration Plug-Ins and Filter

To communicate with the devices that do not support the T10 SCSI standard, your host uses a single VAAI filter and a vendor-specific VAAI plug-in. Use the esxcli command to view the hardware acceleration filter and plug-ins currently loaded into your system.

Prerequisites

Install ESXCLI. See Getting Started with ESXCLI. For troubleshooting, run esxcli commands in the ESXi Shell.

Procedure

  • ♦ Run the esxcli storage core plugin list --plugin-class=value command.
    For value, enter one of the following parameters:
    • Type VAAI to display plug-ins.
      The output of this command is similar to the following example: 
      #esxcli storage core plugin list --plugin-class=VAAI
Plugin name         Plugin class
VMW_VAAIP_EQL       VAAI
VMW_VAAIP_NETAPP    VAAI
VMW_VAAIP_CX        VAAI
    • Type Filter to display the Filter.
      The output of this command is similar to the following example: 
      esxcli storage core plugin list --plugin-class=Filter
Plugin name  Plugin class 
VAAI_FILTER  Filter

Verify Hardware Acceleration Support Status

Use the esxcli command to verify the hardware acceleration support status of a particular storage device.

Prerequisites

Install ESXCLI. See Getting Started with ESXCLI. For troubleshooting, run esxcli commands in the ESXi Shell.

Procedure

  • ♦ Run the esxcli storage core device list -d=device_ID command.
    The output shows the hardware acceleration, or VAAI, status that can be unknown, supported, or unsupported. 
    # esxcli storage core device list -d naa.XXXXXXXXXXXX4c
naa.XXXXXXXXXXXX4c
 Display Name: XXXX Fibre Channel Disk(naa.XXXXXXXXXXXX4c)
 Size: 20480
 Device Type: Direct-Access
 Multipath Plugin: NMP
 XXXXXXXXXXXXXXXX
 Attached Filters: VAAI_FILTER
 VAAI Status: supported
 XXXXXXXXXXXXXXXX

Verify Hardware Acceleration Support Details

Use the esxcli command to query whether the block storage device provides the hardware acceleration support.

Prerequisites

Install ESXCLI. See Getting Started with ESXCLI. For troubleshooting, run esxcli commands in the ESXi Shell.

Procedure

  • ♦ Run the esxcli storage core device vaai status get -d=device_ID command.
    If a VAAI plug-in manages the device, the output shows the name of the plug-in attached to the device. The output also shows the support status for each T10 SCSI based primitive, if available. Output appears in the following example: 
    # esxcli storage core device vaai status get -d naa.XXXXXXXXXXXX4c
naa.XXXXXXXXXXXX4c
 VAAI Plugin Name: VMW_VAAIP_SYMM
 ATS Status: supported
 Clone Status: supported
 Zero Status: supported
 Delete Status: unsupported

List Hardware Acceleration Claim Rules

Each block storage device managed by a VAAI plug-in needs two claim rules. One claim rule specifies the hardware acceleration filter and another specifies the hardware acceleration plug-in for the device. You can use the esxcli commands to list the hardware acceleration filter and plug-in claim rules.

Procedure

  1. To list the filter claim rules, run the esxcli storage core claimrule list --claimrule-class=Filter command.
    In this example, the filter claim rules specify devices that the VAAI_FILTER filter claims. 
    # esxcli storage core claimrule list --claimrule-class=Filter
Rule Class   Rule   Class   Type   Plugin      Matches                    XCOPY Use Array Reported Values  XCOPY Use Multiple Segments  XCOPY Max Transfer Size KiB
Filter       65430  runtime vendor VAAI_FILTER vendor=EMC model=SYMMETRIX                           False                        False                            0
Filter       65430  file    vendor VAAI_FILTER vendor=EMC model=SYMMETRIX                           False                        False                            0
Filter       65431  runtime vendor VAAI_FILTER vendor=DGC model=*                                   False                        False                            0
Filter       65431  file    vendor VAAI_FILTER vendor=DGC model=*                                   False                        False                            0
  2. To list the VAAI plug-in claim rules, run the esxcli storage core claimrule list --claimrule-class=VAAI command.
    In this example, the VAAI claim rules specify devices that the VAAI plug-in claims. 
    esxcli storage core claimrule list --claimrule-class=VAAI
Rule Class   Rule   Class   Type    Plugin         Matches                   XCOPY Use Array Reported Values  XCOPY Use Multiple Segments  XCOPY Max Transfer Size KiB
VAAI         65430  runtime vendor  VMW_VAAIP_SYMM vendor=EMC model=SYMMETRIX                          False                        False                            0
VAAI         65430  file    vendor  VMW_VAAIP_SYMM vendor=EMC model=SYMMETRIX                          False                        False                            0
VAAI         65431  runtime vendor  VMW_VAAIP_CX   vendor=DGC model=*                                  False                        False                            0
VAAI         65431  file    vendor  VMW_VAAIP_CX   vendor=DGC model=*                                  False                        False                            0

Add Hardware Acceleration Claim Rules

To configure the hardware acceleration for a new array, add two claim rules, one for the VAAI filter and another for the VAAI plug-in. For the new claim rules to be active, you first define the rules and then load them into your system.

Prerequisites

Install ESXCLI. See Getting Started with ESXCLI. For troubleshooting, run esxcli commands in the ESXi Shell.

Procedure

  1. Define a new claim rule for the VAAI filter by running the esxcli storage core claimrule add --claimrule-class=Filter --plugin=VAAI_FILTER command.
  2. Define a new claim rule for the VAAI plug-in by running the esxcli storage core claimrule add --claimrule-class=VAAI command.
  3. Load both claim rules by running the following commands:
    esxcli storage core claimrule load --claimrule-class=Filter

    esxcli storage core claimrule load --claimrule-class=VAAI

  4. Run the VAAI filter claim rule by using the esxcli storage core claimrule run --claimrule-class=Filter command.
    Note: Only the filter-class rules must be run. When the VAAI filter claims a device, it automatically finds the proper VAAI plug-in to attach.

Example: Defining Hardware Acceleration Claim Rules

This example shows how to configure the hardware acceleration for IBM arrays using the VMW_VAAIP_T10 plug-in. Use the following sequence of commands. For information about the options that the command takes, see Add Multipathing Claim Rules.

# esxcli storage core claimrule add --claimrule-class=Filter --plugin=VAAI_FILTER --type=vendor --vendor=IBM --autoassign

# esxcli storage core claimrule add --claimrule-class=VAAI --plugin=VMW_VAAIP_T10 --type=vendor --vendor=IBM --autoassign

# esxcli storage core claimrule load --claimrule-class=Filter

# esxcli storage core claimrule load --claimrule-class=VAAI

# esxcli storage core claimrule run --claimrule-class=Filter

Configure XCOPY Parameters

XCOPY is one of the VAAI primitives that is used for offloading tasks to the storage array. For example, you can use XCOPY to offload such operations as migration or cloning of virtual machines to the array instead of consuming vSphere resources to perform these tasks.

You can use the XCOPY mechanism with all storage arrays that support the SCSI T10 based VMW_VAAIP_T10 plug-in developed by VMware. To enable the XCOPY mechanism, create a claim rule of the VAAI class.

Prerequisites

Install ESXCLI. See Getting Started with ESXCLI. For troubleshooting, run esxcli commands in the ESXi Shell.

Procedure

  • ♦ Use the following command and enter the XCOPY options:
    esxcli storage core claimrule add --claimrule-class=VAAI

    For information about the options that the command takes, see Add Multipathing Claim Rules.

    Option Description
    -a|--xcopy-use-array-values Use array reported values for XCOPY commands.
    -s|--xcopy-use-multi-segs Use multiple segments for XCOPY commands. Valid only when --xcopy-use-array-values is specified.
    -m|--xcopy-max-transfer-size Maximum transfer size in MB for the XCOPY commands when you use a transfer size different than array reported. Valid only when --xcopy-use-array-values is specified.
    -k|--xcopy-max-transfer-size-kib Maximum transfer size in KiB for the XCOPY commands when you use a transfer size different than array reported. Valid only if --xcopy-use-array-values is specified.

    -e | --extended-xcopy-support

    		 Use the extended XCOPY command to optimize the data copy between datastores that are in different storage arrays.

Example: Configuring XCOPY

  • # esxcli storage core claimrule add -r 914 -t vendor -V XtremIO -M XtremApp -P VMW_VAAIP_T10 -c VAAI -a -s -k 64
  • # esxcli storage core claimrule add -r 65430 -t vendor -V EMC -M SYMMETRIX -P VMW_VAAIP_SYMM -c VAAI -a -s -m 200

Optimize Data Copying with Extended XCOPY Parameter

The extended XCOPY parameter optimizes the data copy between datastores across two different arrays. It allows you to migrate and clone workloads to offload XCOPY between two different arrays.

Extended XCOPY characteristics:
  • Use a new extended XCOPY option in a claim rule.

    esxcli storage core claimrule add -t vendor -V vendor -M model name -P VMW_VAAIP_T10 -c VAAI --autoassign -e

    Example:

    esxcli storage core claimrule add -t vendor -V DellEMC -M PowerStore -P VMW_VAAIP_T10 -c VAAI --autoassign -e

  • Extended XCOPY is supported only on T10 plug-in.
Procedure
  1. Use the following command to check the extended XCOPY status.
    esxcli storage core device vaai status get -d device
    Example: 
    [root@tpp-h127:~] esxcli storage core device vaai status get -d naa.68ccf098007f554c61148eb1786e6749
naa.68ccf098007f554c61148eb1786e6749
   VAAI Plugin Name: VMW_VAAIP_T10
   ATS Status: supported
   Clone Status: supported
   Zero Status: supported
   Delete Status: supported
   Ex Clone Status: supported"
  2. Enable extended XCOPY for the supported storage array.
    1. Verify if the VAAI_FILTER plug-in device is added.
      esxcli storage core claimrule list -c Filter
    2. If the device is not added for VAAI_FILTER plug-in, use the following command.
      esxcli storage core claimrule add -t vendor -V vendor -M model name -P VAAI_FILTER -c Filter --autoassign
    3. Load the VAAI_FILTER claim rule.
      esxcli storage core claimrule load -c Filter
    4. Add the extended XCOPY claim rule with T10 plug-in.
      esxcli storage core claimrule add -t vendor -V vendor -M model name -P VMW_VAAIP_T10 -c VAAI --autoassign -e
      Note:
      • The claim rule in step d uses ESXi default transfer size and single segment for XCOPY commands. To use array reported values of different transfer size and multiple segments, use optional parameters -a, -s, -m|-k with the above command.
      • Extended XCOPY setup works across different arrays.
      • Add a claim rule with optional parameters such as -a, -s, -m|-k for both source and destination devices. In this case, the minimum transfer size for source and destination arrays will be considered for data transfer.

        Source Device:

        esxcli storage core claimrule add -t vendor -V vendor -M model name -P VMW_VAAIP_T10 -c VAAI --autoassign -a -s

        Destination Device:

        esxcli storage core claimrule add -t vendor -V vendor -M model name -P VMW_VAAIP_T10 -c VAAI --autoassign -a -s -e
    5. Load the VAAI_T10 plug-in.
      esxcli storage core claimrule load -c VAAI
    6. Run the newly added claim rule options.
      esxcli storage core claimrule run --claimrule-class=Filter

Delete Hardware Acceleration Claim Rules

Use the esxcli command to delete existing hardware acceleration claim rules.

Prerequisites

Install ESXCLI. See Getting Started with ESXCLI. For troubleshooting, run esxcli commands in the ESXi Shell.

Procedure

  • ♦ Run the following commands:
    esxcli storage core claimrule remove -r claimrule_ID --claimrule-class=Filter

    esxcli storage core claimrule remove -r claimrule_ID --claimrule-class=VAAI