Using Persistent Memory Devices with ESXi

ESXi supports next generation persistent memory devices, also known as Non-Volatile Memory (NVM) devices. These devices combine performance and speed of memory with the persistence of traditional storage. They can retain stored data through reboots or power source failures.

Virtual machines that require high bandwidth, low latency, and persistence can benefit from this technology. Examples include VMs with acceleration databases and analytics workload.

To use persistent memory with your ESXi host, you must be familiar with the following concepts.

PMem Datastore

After you add persistent memory to your ESXi host, the host detects the hardware, and then formats and mounts it as a local PMem datastore. ESXi uses VMFS-L as a file system format. Only one local PMem datastore per host is supported.

Note: When you manage physical persistent memory, make sure to evacuate all VMs from the host and place the host into maintenance mode.

To reduce administrative overhead, the PMem datastore offers a simplified management model. Traditional datastore tasks do not generally apply to the datastore because the host automatically performs all the required operations on the background. As an administrator, you cannot display the datastore in the Datastores view of the vSphere Client, or perform other regular datastore actions. The only operation available to you is monitoring statistics for the PMem datastore.

The PMem datastore is used to store virtual NVDIMM devices and traditional virtual disks of a VM. The VM home directory with the vmx and vmware.log files cannot be placed on the PMem datastore.

PMem Access Modes

ESXi exposes persistent memory to a VM in two different modes. PMem-aware VMs can have direct access to persistent memory. Traditional VMs can use fast virtual disks stored on the PMem datastore.

Direct-Access Mode

In this mode, also called virtual PMem (vPMem) mode, a PMem region can be presented to a VM as a virtual non-volatile dual in-line memory module (NVDIMM) module. The VM uses the NVDIMM module as a standard byte-addressable memory that can persist across power cycles.

You can add one or several NVDIMM modules when provisioning the VM.

The VMs must be of the hardware version ESXi 6.7 or later and have a PMem-aware guest OS. The NVDIMM device is compatible with latest guest OSes that support persistent memory, for example, Windows 2016.

Each NVDIMM device is automatically stored on the PMem datastore.

Virtual Disk Mode

This mode, also called virtual PMem disks (vPMemDisk) mode, is available to any traditional VM and supports any hardware version, including all legacy versions. VMs are not required to be PMem-aware. When you use this mode, you create a regular SCSI virtual disk and attach a PMem VM storage policy to the disk. The policy automatically places the disk on the PMem datastore.

PMem Storage Policy

To place the virtual disk on the PMem datastore, you must apply the host-local PMem default storage policy to the disk. The policy is not editable.

The policy can be applied only to virtual disks. Because the VM home directory does not reside on the PMem datastore, make sure to place it on any standard datastore.

After you assign the PMem storage policy to the virtual disk, you cannot change the policy through the VM Edit Setting dialog box. To change the policy, migrate or clone the VM.

The following graphic illustrates how the persistent memory components interact.

For information about how to configure and manage VMs with NVDIMMs or virtual persistent memory disks, see the vSphere Resource Management documentation and vSphere Virtual Machine Administration.

Monitor PMem Datastore Statistics

You can use the vSphere Client and the esxcli command to review the capacity of the PMem datastore and some of its other attributes.

However, unlike regular datastores, such as VMFS or vVol, the PMem datastore does not appear in the Datastores view of the vSphere Client. Regular datastore administrative tasks do not apply to it.

Procedure

• ♦ Review PMem datastore information.

  Option	Description
  vSphere Client	Navigate to the ESXi host and click Summary. In the Hardware panel, verify that Persistent Memory is displayed and review its capacity.
  esxcli command	Use the esxcli storage filesystem list to list the PMem datastore.

Example: Viewing the PMem Datastore

The following sample output appears when you use the esxcli storage filesystem list command to list the datastore.

# esxcli storage filesystem list
Mount Point             Volume Name         UUID         Mounted   Type      Size           Free
----------------------  ----------------  -------------  -------- ------- ------------  ------------

/vmfs/volumes/5xxx...     ds01-102         5xxx...       true     VMFS-6  14227079168   12718178304
/vmfs/volumes/59ex...     ds02-102         59ex...       true     VMFS-6  21206401024   19697500160
/vmfs/volumes/59bx...                      59bx...       true     vfat    4293591040    4274847744
/vmfs/volumes/pmem:5ax... PMemDS-56ax...   pmem:5a0x...  true     PMEM    12880707584   11504975872