vSphere Network I/O Control Design for a Virtual Infrastructure Workload Domain

Use vSphere Network I/O Control to allocate network bandwidth to customer workloads and to resolve situations where several types of traffic compete for shared network bandwidth.

When Network I/O Control is enabled, the distributed switch allocates bandwidth for the traffic that is related to the main vSphere features.

• Fault tolerance traffic

• iSCSI traffic

• vSphere vMotion traffic

• Management traffic

• VMware vSphere Replication traffic

• NFS traffic

• vSAN traffic

• Backup traffic

• Virtual machine traffic

Network I/O Control Heuristics

The following heuristics can help with design decisions for Network I/O Control.

Shares and Limits

When you use bandwidth allocation, consider using shares instead of limits. Limits impose hard limits on the amount of bandwidth used by a traffic flow even when network bandwidth is available.

Limits on Network Resource Pools

Consider imposing limits on a given network resource pool. For example, if you put a limit on vSphere vMotion traffic, you can benefit in situations where multiple vSphere vMotion data transfers, initiated on different ESXi hosts at the same time, result in oversubscription at the physical network level. By limiting the available bandwidth for vSphere vMotion at the ESXi host level, you can prevent performance degradation for other traffic.

Teaming Policy

When you use Network I/O Control, use Route based on physical NIC load teaming as a distributed switch teaming policy to maximize the networking capacity utilization. With load-based teaming, traffic might move among uplinks, and reordering of packets at the receiver can result occasionally.

Traffic Shaping

Use distributed port groups to apply configuration policies to different traffic types. Traffic shaping can help in situations where multiple vSphere vMotion migrations initiated on different ESXi hosts converge on the same destination ESXi host. The actual limit and reservation also depend on the traffic shaping policy for the distributed port group where the adapter is connected to.

How Network I/O Control Works

Network I/O Control enforces the share value specified for the different traffic types when a network contention occurs. Network I/O Control applies the share values set to each traffic type. As a result, less important traffic, as defined by the share percentage, is throttled, granting access to more network resources to more important traffic types.

Network I/O Control also supports reservation of bandwidth for system traffic based on the capacity of the physical adapters on an ESXi host and enables fine-grained resource control at the virtual machine network adapter level. Resource control is similar to the model for CPU and memory reservations in vSphere DRS.

Table 1. Design Decisions on vSphere Network I/O Control

Decision ID	Design Decision	Design Justification	Design Implication
VCF-WLD-VCS-VDS-007	Enable Network I/O Control on the vSphere Distributed Switch for the VI workload domain cluster.	Increases resiliency and performance of the network.	If configured incorrectly, Network I/O Control might impact network performance for critical traffic types.
VCF-WLD-VCS-VDS-008	Set the share value for management traffic to Normal.	By keeping the default setting of Normal, management traffic is prioritized higher than vSphere vMotion but lower than vSAN traffic. Management traffic is important because it ensures that the hosts can still be managed during times of network contention.	None.
VCF-WLD-VCS-VDS-009	Set the share value for vSphere vMotion traffic to Low.	During times of network contention, vSphere vMotion traffic is not as important as virtual machine or storage traffic.	During times of network contention, a vSphere vMotion migration takes longer than usual to complete.
VCF-WLD-VCS-VDS-010	Set the share value for virtual machines to High.	Virtual machines are the most important asset in the SDDC. Leaving the default setting of High ensures that they always have access to the network resources they need.	None.
VCF-WLD-VCS-VDS-011	Set the share value for vSphere Fault Tolerance to Low.	This design does not use vSphere Fault Tolerance. Fault tolerance traffic can be set the lowest priority.	None.
VCF-WLD-VCS-VDS-012	Set the share value for the principal storage traffic type, for example, vSAN, to High.	During times of network contention, principal storage traffic needs a guaranteed bandwidth to support virtual machine performance.	None.
VCF-WLD-VCS-VDS-013	Set the share value for backup traffic to Low.	During times of network contention, the primary functions of the SDDC must continue to have access to network resources with priority over backup traffic.	During times of network contention, backups are slower than usual.
VCF-WLD-VCS-VDS-014	Set the share value for other traffic types to Low.	By default, VMware Cloud Foundation does not use other traffic types, like vSphere Fault Tolerance. Hence, these traffic types can be assigned the lowest priority.	None.