Virtual network interrupt coalescing can reduce the number of interrupts, thus potentially decreasing CPU utilization. Though this could increase network latency, many workloads are not impacted by additional network latency of anywhere from a few hundred microseconds to a few milliseconds, and the reduction in virtual networking overhead can potentially allow more virtual machines to be run on a single VMware Cloud on AWS host.
By default, this feature is activated for all virtual NICs in VMware Cloud on AWS. For VMXNET3 virtual NICs, however, this feature can be set to one of three schemes or deactivated by changing the ethernetX.coalescingScheme variable (where X is the number of the virtual NIC to configure).
The feature will be activated (the default) if the ethernetX.coalescingScheme variable is not present or if the variable is present and set to rbc (which stands for rate-based coalescing).
In this case, the ethernetX.coalescingParams variable can be used to set the virtual network interrupt rate in interrupts per second. The value can range from 100 to 100,000, with a default of 4,000.
The feature can be set to queue a predefined number of packets before interrupting the virtual machine or transmitting the packets by setting ethernetX.coalescingScheme to static.
In this case, the ethernetX.coalescingParams variable can be used to set the number of packets VMware Cloud on AWS will queue. The value can range from 1 to 64, with a default of 64. A larger queue size can reduce the number of context switches between the virtual machine and the VMkernel, potentially reducing CPU utilization both in the virtual machine and in the VMkernel.
Regardless of the number of packets queued, however, VMware Cloud on AWS waits no longer than 4 milliseconds before sending an interrupt or transmitting the packets. Other events, such as the virtual machine being idle, can also trigger virtual machine interrupts or packet transmission, so packets are rarely delayed the full 4 milliseconds.
The feature can be set to be adaptive by setting ethernetX.coalescingScheme to adapt. This setting bases the interrupt rate on both the virtual machine load and the overall system load; it uses a lower interrupt rate when the load is high and a higher interrupt rate when the load is low.
The feature can be deactivated by setting ethernetX.coalescingScheme to disabled. Deactivating this feature will typically result in more interrupts (and thus higher CPU utilization), but will typically also lead to lower network latency.
To configure VMXNET3 virtual interrupt coalescing through the vSphere Client:
Right-click the virtual machine you wish to change, then select Edit Settings.
Under the VM Options tab, expand Advanced, then click EDIT CONFIGURATION.
Look for and set the variable you wish to change:
If selecting a virtual interrupt coalescing scheme, look for ethernetX.coalescingScheme (where X is the number of the virtual NIC to configure) and set it to your desired value.
If the variable isn’t present, click ADD CONFIGURATION PARAMS and enter it as a new parameter.
If configuring the virtual network interrupt rate or queue size, look for ethernetX.coalescingParams (where X is the number of the virtual NIC to configure) and set it to your desired value.
If the variable isn’t present, click ADD CONFIGURATION PARAMS and enter it as a new parameter.
Click OK to close the Configuration Parameters window.
Click OK to close the Edit Settings window.
The change will not take effect until the virtual machine has been restarted.
