This section describes various virtual network adapter features and how to configure the adapters for the best performance.

  • When networking two virtual machines on the same host, try to connect them to the same vSwitch. When connected this way, their network speeds are not limited by the wire speed of any physical network card. Instead, they transfer network packets as fast as the host resources allow.

  • Jumbo frames are recommended as a way to increase network throughout and reduce CPU load; they do this by allowing data to be transmitted using larger, and therefore fewer, packets.

    Jumbo frames are supported on the E1000, E1000E, VMXNET2, and VMXNET3 devices. They are activated by default on the underlying network for all same-data-center traffic, Connected VPC traffic, and VMware Transit Connect™ traffic. For paths using Direct Connect, jumbo frames must be explicitly activated on the Direct Connect private virtual interface by following the steps in Specify the Direct Connect MTU. The maximum supported MTU sizes are listed on the VMware Configuration Maximums site (Select Product: VMware Cloud on AWS, select your version, expand Networking and Security, select General Networking, click VIEW LIMITS).

  • TCP Segmentation Offload (TSO) in VMware Cloud on AWS is activated by default in the VMkernel but is supported in virtual machines only when they are using an E1000, E1000E, VMXNET2, or VMXNET3 device. TSO can improve performance even if the underlying hardware does not support TSO.

  • Similarly, Large Receive Offload (LRO) in VMware Cloud on AWS is activated by default in the VMkernel, but is supported in virtual machines only when they are using the VMXNET2 or VMXNET3 device.

    Note:

    LRO support varies by operating system. Many Linux variants support LRO. In Windows virtual machines LRO is supported when the following prerequisites are met:

    • The virtual machine uses virtual hardware version 11 or later.

    • The virtual machine uses the VMXNET3 device.

    • The virtual machine is running Windows Server 2012 or later or Windows 8.0 or later.

    • The guest operating system uses VMXNET3 vNIC driver version 1.6.6.0 or later.

    • Receive Segment Coalescing (RSC) is globally activated in the guest operating system.

    For more information about configuring LRO, see the Large Receive Offload section of vSphere Networking.

  • In some cases, low receive throughput in a virtual machine can be due to insufficient receive buffers (also described as an overflowing receive ring) in the receiver network device. If the receive buffers in the guest operating system’s network driver are insufficient, packets will be dropped in the VMkernel, degrading network throughput. A possible workaround is to increase the number of receive buffers, though this might increase the host physical CPU workload.

    For VMXNET, the default number of receive and transmit buffers is 100 each, with the maximum possible being 128. For VMXNET2, the default number of receive and transmit buffers are 150 and 256, respectively, with the maximum possible receive buffers being 512. See VMware KB article 1010071 for information about altering these defaults.

    For E1000, E1000E, and VMXNET3, the default number of receive and transmit buffers are controlled by the guest driver, with the maximum possible for both being 4096. In Linux, these values can be changed from within the guest by using ethtool. In Windows, the values can be changed from within the guest in the device properties window. For additional information see VMware KB article 1010071.

  • Multiple receive and transmit queues (often referred to as receive-side scaling (RSS) or scalable I/O) allow network packets from a single NIC to be scheduled in parallel on multiple CPUs. Without multiple queues, network interrupts can be handled on only one CPU at a time. Multiple queues help throughput in cases where a single CPU would otherwise be saturated with network packet processing and become a bottleneck. To prevent out-of-order packet delivery, the driver schedules all of a flow’s packets to the same CPU.

    The E1000E and VMXNET3 devices support multiple queues for many guest operating systems that natively support the feature, which include Windows Server 2003 SP2 and later, Windows 7 and later, and some Linux distributions.

    The VMXNET3 drivers included with the vSphere 5.0 and later versions of VMware Tools default to having multiple receive queues activated, as does the VMXNET3 driver included with Linux kernel 2.6.37 and later. For more information, see VMware KB article 2020567.

    When multiple receive queues are activated, the feature by default configures 1, 2, 4, or 8 receive queues in a virtual machine, choosing the largest of these values less than or equal to the number of vCPUs in that virtual machine.

    In both Windows and Linux the number the number of transmit queues, by default, is the same as the number of receive queues.

    In order to obtain the maximum performance with your specific workloads and resource availability you can try out different values for the number of receive queues (which must be set to a power of 2 and can be a maximum of 8 or the number of vCPUs, whichever is lower). This setting is changed on the advanced driver configuration tab within the guest operating system.