Networking Best Practices for vSphere vMotion

Consider certain best practices for configuring the network resources for vMotion on an ESXi host.

Provide the required bandwidth in one of the following ways:

Physical Adapter Configuration	Best Practices
Dedicate at least one adapter for vMotion or dedicate network bandwidth for vMotion	Use at least one 10 GbE adapter for workloads that have a small number of memory operations. Alternatively, you can reserve at least 10 GbE of network bandwidth for vMotion on a high speed link. Use at least one 25 GbE adapter if you migrate workloads that have many memory operations. Alternatively, you can reserve at least 25 GbE of network bandwidth for vMotion on a high speed link. If only two Ethernet adapters are available, for best availability, combine both adapters into a team, and use VLANs to divide traffic into networks: one or more for virtual machine traffic and one for vMotion.
Direct vMotion traffic to one or more physical NICs that have high-bandwidth capacity and are shared between other types of traffic as well	To distribute and allocate more bandwidth to vMotion traffic across several physical NICs, use multiple-NIC vMotion. On a vSphere Distributed Switch 5.1 and later, use vSphere Network I/O Control shares to guarantee bandwidth to outgoing vMotion traffic. Defining shares also prevents from contention as a result from excessive vMotion or other traffic. To avoid saturation of the physical NIC link as a result of intense incoming vMotion traffic, use traffic shaping in egress direction on the vMotion port group on the destination host. By using traffic shaping you can limit the average and peak bandwidth available to vMotion traffic, and reserve resources for other traffic types. In vSphere 7.0 Update 1 or earlier, vMotion saturates 1 GbE and 10 GbE physical NICs with a single vMotion VMkernel NIC. Starting with vSphere 7.0 Update 2, vMotion saturates high speed links such as 25 GbE, 40 GbE and 100 GbE with a single vMotion VMkernel NIC. If you do not have dedicated uplinks for vMotion, you can use Network I/O Control to limit vMotion bandwidth use.

Physical Adapter Configuration

Best Practices

Dedicate at least one adapter for vMotion or dedicate network bandwidth for vMotion

Use at least one 10 GbE adapter for workloads that have a small number of memory operations. Alternatively, you can reserve at least 10 GbE of network bandwidth for vMotion on a high speed link.

Use at least one 25 GbE adapter if you migrate workloads that have many memory operations. Alternatively, you can reserve at least 25 GbE of network bandwidth for vMotion on a high speed link.

If only two Ethernet adapters are available, for best availability, combine both adapters into a team, and use VLANs to divide traffic into networks: one or more for virtual machine traffic and one for vMotion.

Direct vMotion traffic to one or more physical NICs that have high-bandwidth capacity and are shared between other types of traffic as well

To distribute and allocate more bandwidth to vMotion traffic across several physical NICs, use multiple-NIC vMotion.
On a vSphere Distributed Switch 5.1 and later, use vSphere Network I/O Control shares to guarantee bandwidth to outgoing vMotion traffic. Defining shares also prevents from contention as a result from excessive vMotion or other traffic.
To avoid saturation of the physical NIC link as a result of intense incoming vMotion traffic, use traffic shaping in egress direction on the vMotion port group on the destination host. By using traffic shaping you can limit the average and peak bandwidth available to vMotion traffic, and reserve resources for other traffic types.
In vSphere 7.0 Update 1 or earlier, vMotion saturates 1 GbE and 10 GbE physical NICs with a single vMotion VMkernel NIC. Starting with vSphere 7.0 Update 2, vMotion saturates high speed links such as 25 GbE, 40 GbE and 100 GbE with a single vMotion VMkernel NIC. If you do not have dedicated uplinks for vMotion, you can use Network I/O Control to limit vMotion bandwidth use.

Use jumbo frames for best vMotion performance.
Ensure that jumbo frames are enabled on all network devices that are on the vMotion path including physical NICs, physical switches, and virtual switches.
Place vMotion traffic on the vMotion TCP/IP stack for migration across IP subnets that have a dedicated default gateway that is different from the gateway on the management network. See Place vMotion Traffic on the vMotion TCP/IP Stack of an ESXi Host.

For information about configuring networking on an ESXi host, see the vSphere Networking documentation.