Use flow control to manage the rate of data transfer between senders and receivers on the vSAN network. Congestion control handles congestion in the network.
Flow Control
You can use flow control to manage the rate of data transfer between two devices.
Flow control is configured when two physically connected devices perform auto-negotiation.
An overwhelmed network node might send a pause frame to halt the transmission of the sender for a specified period. A frame with a multicast destination address sent to a switch is forwarded out through all other ports of the switch. Pause frames have a special multicast destination address that distinguishes them from other multicast traffic. A compliant switch does not forward a pause frame. Frames sent to this range are meant to be acted upon only within the switch. Pause frames have a limited duration, and expire after a time interval. Two computers that are connected through a switch never send pause frames to each other, but can send pause frames to a switch.
One reason to use pause frames is to support network interface controllers (NICs) that do not have enough buffering to handle full-speed reception. This problem is uncommon with advances in bus speeds and memory sizes.
Congestion Control
Congestion control helps you control the traffic on the network.
Congestion control applies mainly to packet switching networks. Network congestion within a switch might be caused by overloaded inter-switch links. If inter-switch links overload the capability on the physical layer, the switch introduces pause frames to protect itself.
Priority Flow Control
Priority-based flow control (PFC) helps you eliminate frame loss due to congestion.
Priority-based flow control (IEEE 802.1Qbb) is achieved by a mechanism similar to pause frames, but operates on individual priorities. PFC is also called Class-Based Flow Control (CBFC) or Per Priority Pause (PPP).
Flow Control and Congestion Control
Flow control is an end-to-end mechanism that controls the traffic between a sender and a receiver. Flow control occurs in the data link layer and the transport layer.
Congestion control is used by a network to control congestion in the network. This problem is not as common in modern networks with advances in bus speeds and memory sizes. A more likely scenario is network congestion within a switch. Congestion Control is handled by the network layer and the transport layer.
Flow Control Design Considerations
By default, flow control is enabled on all network interfaces in ESXi hosts.
Flow control configuration on a NIC is done by the driver. When a NIC is overwhelmed by network traffic, the NIC sends pause frames.
Flow control mechanisms such as pause frames can trigger overall latency in the VM guest I/O due to increased latency at the vSAN network layer. Some network drivers provide module options that configure flow control functionality within the driver. Some network drivers enable you to modify the configuration options using the ethtool command-line utility on the console of the ESXi host. Use module options or ethtool, depending on the implementation details of a given driver.
For information about configuring flow control on ESXi hosts, see VMware KB 1013413.
In deployments with 1 Gbps, leave flow control enabled on ESXi network interfaces (default). If pause frames are a problem, carefully plan disabling flow control in conjunction with Hardware Vendor Support or VMware Global Support Services.
To learn how you can recognize the presence of pause frames being sent from a receiver to an ESXi host, see Troubleshooting the vSAN Network. A large number of pause frames in an environment usually indicates an underlying network or transport issue to investigate.
