Virtualization causes an increase in the amount of physical memory required due to the extra memory needed by VMware Cloud on AWS for its own code and for data structures.

This additional memory requirement can be separated into two components:

  1. A system-wide memory space overhead for the VMkernel and various host agents (hostd, vpxa, vSAN, etc.).

  2. An additional memory space overhead for each virtual machine.

    The per-virtual-machine memory space overhead can be further divided into the following categories:

    • Memory reserved for the virtual machine executable (VMX) process.

      This is used for data structures needed to bootstrap and support the guest (i.e., thread stacks, text, and heap).

    • Memory reserved for the virtual machine monitor (VMM).

      This is used for data structures required by the virtual hardware (i.e., TLB, memory mappings, and CPU state).

    • Memory reserved for various virtual devices (i.e., mouse, keyboard, SVGA, USB, etc.).

    • Memory reserved for other subsystems, such as the kernel, management agents, etc.

    The amounts of memory reserved for these purposes depend on a variety of factors, including the number of vCPUs, the configured memory for the guest operating system, whether the guest operating system is 32-bit or 64-bit, the VMM execution mode, and which features are activated for the virtual machine. For more information about these overheads, see vSphere Resource Management.

    While the VMM and virtual device memory needs are fully reserved at the time the virtual machine is powered on, a feature called VMX swap can significantly reduce the per-virtual-machine VMX memory reservation, allowing more memory to be swapped out when host memory is overcommitted. This represents a significant reduction in the overhead memory reserved for each virtual machine.

    The creation of a VMX swap file for each virtual machine (and thus the reduction in host memory reservation for that virtual machine) is automatic. By default, this file is created in the virtual machine’s working directory, but a different location can be set with sched.swap.vmxSwapDir.

    The amount of disk space required varies, but even for a large virtual machine is typically less than 100MB (typically less than 300MB if the virtual machine is configured for 3D graphics). The system retains a small working set memory reservation, and does not swap under ordinary usage. Extraordinary usage (for example, taking screenshots every second, or collecting a support dump) might cause swapping.

    Note:

    The VMX swap file is entirely unrelated to swap to host-level swapping, which is described in Memory Overcommit Techniques.

In addition, VMware Cloud on AWS also provides optimizations, such as page sharing (see Memory Overcommit Techniques), to reduce the amount of physical memory used on the underlying server. In some cases these optimizations can save more memory than is taken up by the overhead described in this section.