Manage Persistent Memory

ESXi 6.7 and later provides support for the latest computer memory technology, which is called non-volatile memory (NVM) or persistent memory (PMem).

PMem combines the high data transfer rate of volatile computer memory with the persistence and resiliency of traditional storage. PMem devices have low access latency and can retain stored data through reboots or power outages.

Modes of Consumption of the Persistent Memory Resources of the Host

When you add a physical PMem device to a host, ESXi detects the PMem resource and exposes it as a host-local PMem datastore to the virtual machines that run on the host. Depending on the guest operating system, virtual machines can have direct access to the PMem resources.

Each host can have only one local PMem datastore that pools and represents all PMem resources of the host.

Persistent memory combines the properties of both memory and storage. So, virtual machines can consume the PMem resources of the ESXi host as memory (through virtual NVDIMM devices) or as storage (through virtual PMem hard disks).

The host-local PMem datastore stores all direct-accessed NVDIMM devices and virtual PMem hard disks.

Virtual PMem (vPMem)

In this mode, if the guest operating system is PMem-aware, the virtual machine can have direct access to the physical PMem resources of the host and use them as standard, byte-addressable memory.

Virtual machines use virtual non-volatile dual in-line memory modules (NVDIMMs) for direct access to PMem. The NVDIMM is a memory device that sits on an ordinary memory channel, but contains non-volatile memory. In vSphere 7.0, the virtual NVDIMM is a new type of device that represents the physical PMem regions of the host. A single virtual machine can have up to 64 virtual NVDIMM devices. Each NVDIMM device is stored on the host-local PMem datastore.

Note: To add an NVDIMM device to a virtual machine, the virtual machine must be of hardware version 14 and the guest operating system must support persistent memory. If the guest operating system is not PMem-aware, you can still use PMem, but you cannot add an NVDIMM device to the virtual machine.

Virtual PMem Disks (vPMemDisk)

In this mode, the virtual machine does not have direct access to the PMem resources of the host.

You must add a virtual PMem hard disk to the virtual machine. A virtual PMem hard disk is a traditional SCSI disk to which the PMem Storage Policy is applied. The policy automatically places the hard disk on the host-local PMem datastore.

In this mode of usage, there are no requirements for the hardware version of the virtual machine and the guest operating system.

Note: If the guest operating system is not PMem-aware, virtual machines can use PMem only through vPMemDisks.

The following diagram illustrates how the persistent memory components interact.

Interaction between persistent memory components

For information about how to configure and manage VMs with NVDIMMs or virtual persistent memory disks, see the vSphere Resource Management documentation.

Add an NVDIMM Device to a Virtual Machine

Add a virtual NVDIMM device to a virtual machine to enable it to use non-volatile, or persistent, computer memory. Non-volatile memory (NVM), or persistent memory (PMem), combines the high data transfer rates of the volatile memory with the persistence and resiliency of traditional storage. The virtual NVDIMM device is a virtual NVM device that can retain stored data through reboots or power source failures.

If an ESXi host fails or the datastore is no longer accessible, when you add an NVDIMM device to a virtual machine, you can configure vSphere HA to failover all PMem virtual machines on another host.

Note: If a host fails, NVDIMM PMem data cannot be restored. vSphere HA will restart the virtual machine on another host with a new and empty NVDIMM of the same size.

When you add an NVDIMM device to a virtual machine, you can take a snapshot of the virtual machine by configuring the NVDIMM snapshot mode of the virtual machine in the independent erase on revert mode or in the independent persistent mode.

NVDIMM Snapshot Conditions and Limitations

You cannot configure the NVDIMM snapshot mode when the virtual machine has existing snapshots.
You cannot add another NVDIMM device to the virtual machine if the NVDIMM snapshot mode is independent persistent or independent erase on revert, and the virtual machine has existing snapshots.
You cannot add an NVDIMM device to a virtual machine, remove an NVDIMM device from a virtual machine, or change the NVDIMM device parameters if the virtual machine has snapshots. To allow these operations, remove all virtual machine snapshots.
You can use vMotion to migrate virtual machines that have snapshots containing NVDIMM devices and revert these snapshots.

For more information, see the vSphere Resource Management guide.

Prerequisites

Power off the virtual machine.
Verify that the guest operating system of the virtual machine supports PMem.
Verify that you have the Datastore.Allocate space privilege on the virtual machine.
Verify that the host or the cluster on which the virtual machine resides has available PMem resources.
To add an NVDIMM device to a virtual machine, verify that the virtual machine hardware is of version 14 or higher.
To configure vSphere HA for PMem virtual machines, verify that the virtual machine is of version 19 or later, and vSphere HA is enabled on the cluster.

Procedure

Add an NVDIMM device to a virtual machine when you deploy a virtual machine or edit an existing virtual machine.

Option	Action
Create a virtual machine	Right-click any inventory object that is a valid parent object of a virtual machine and select New Virtual Machine. On the Select a creation type page, select Create a new virtual machine, and click Next. Navigate through the pages of the wizard. On the Customize hardware page, click the Virtual Hardware tab. On the Virtual Hardware tab, click the Add New Device button. From the drop-down menu, select NVDIMM.
Edit a virtual machine	Right-click a virtual machine in the inventory and select Edit Settings. Click the Virtual Hardware tab. On the Virtual Hardware tab, click the Add New Device button. From the drop-down menu, select NVDIMM.

Option

Action

Create a virtual machine

Right-click any inventory object that is a valid parent object of a virtual machine and select New Virtual Machine.
On the Select a creation type page, select Create a new virtual machine, and click Next.
Navigate through the pages of the wizard.
On the Customize hardware page, click the Virtual Hardware tab.
On the Virtual Hardware tab, click the Add New Device button.
From the drop-down menu, select NVDIMM.

Edit a virtual machine

Right-click a virtual machine in the inventory and select Edit Settings.
Click the Virtual Hardware tab.
On the Virtual Hardware tab, click the Add New Device button.
From the drop-down menu, select NVDIMM.

The NVDIMM device appears in the Virtual Hardware devices list together with the virtual NVDIMM controller. Each virtual machine can have a maximum of one virtual NVDIMM controller and each NVDIMM controller can have up to 64 virtual NVDIMM devices.

Note: You can change the size of the NVDIMM device at a later time. The virtual machine must be powered off.

In the New NVDIMM text box, enter the size of the NVDIMM device and select the units from the drop-down menu.

Note: Adjust the size of the newly added devices in accordance with the amount of the PMem available to the host. If any part of the configuration requires attention, the wizard alerts you.
Expand the New NVDIMM device section and select the Allow failover on another host for all NVDIMM devices check box.

Note: When you allow the failover process, if you add another NVDIMM device to the virtual machine, the NVDIMM device will have PMem HA enabled. If you want to preserve the NVDIMM content of the virtual machine during a host failure, make sure to deselect the Allow failover on another host for all NVDIMM devices check box.

From the Snapshot Mode, configure the NVDIMM snapshot mode.

Option	Description
Persistent	Persistent mode is the default NVDIMM snapshot mode. Taking snapshots of virtual machines with an NVDIMM device in persistent mode is not supported.
Independent - Persistent	Taking snapshots of virtual machines with an NVDIMM device in independent-persistent mode is supported. When you revert to the previous snapshot, the current contents of the NVDIMM device is preserved.
Independent - Erase on revert	Taking snapshots of virtual machines with an NVDIMM device in independent-erase on revert mode is supported. When you revert to the previous snapshot, the content of the NVDIMM device is erased.

If you deploy a virtual machine, click Next.
If you edit an existing virtual machine, click OK.

Results

When you power on the virtual machine, you can view the vSphere HA Protection status of the virtual machine in the vSphere HA panel on the Summary tab.