vSphere SR-IOV support relies on the interaction between the virtual functions (VFs) and the physical function (PF) of the NIC port for better performance, and interaction between the driver of the PF and the host switch for traffic control.

In a host that runs virtual machine traffic on top of SR-IOV physical adapters, virtual machine adapters directly contact the virtual functions to communicate data. However, the ability to configure networks is based on the active policies for the port holding the virtual machines.

On an ESXi host without SR-IOV, the virtual switch sends external network traffic through its ports on the host from or to the physical adapter for the relevant port group. The virtual switch also applies the networking policies on managed packets.

Figure 1. Data and Configuration Paths in the SR-IOV Support of vSphere

The data path and the control path of the SR-IOV support in vSphere involve different components.

The data path directly connects the virtual machine with the virtual function on the NIC. The control path includes the virtual switch and the active policies on the virtual machine.

Data Path in SR-IOV

After the virtual machine network adapter is assigned to a virtual function, the VF driver in the guest operating system uses the I/O memory management unit (IOMMU) technology to access the virtual function that must receive or send the data over the network. The VMkernel, that is, the virtual switch in particular, does not process the data flow, which reduces the overall latency of SR-IOV enabled workloads.

Configuration Path in SR-IOV

If the guest operating system attempts to change the configuration of a virtual machine adapter mapped to a VF, the change takes place if it is allowed by the policy on the port associated with the virtual machine adapter.

The configuration workflow consists of the following operations:

  1. The guest operating system requests a configuration change on the VF.
  2. The VF forwards the request to the PF through a mailbox mechanism.
  3. The PF driver checks the configuration request with the virtual switch (standard switch or host proxy switch of a distributed switch).
  4. The virtual switch verifies the configuration request against the policy on the port with which the VF enabled virtual machine adapter is associated.
  5. The PF driver configures the VF if the new settings are in compliance with the port policy of the virtual machine adapter.

    For example, if the VF driver tries to modify the MAC address, the address remains the same if MAC address change is not allowed in the security policy for the port group or port. The guest operating system might show that the change is successful but a log message indicates that the operation has failed. As a result, the guest operating system and the virtual device save different MAC addresses. The network interface in the guest operating system might not be able to acquire an IP address and communicate. In this case, you have to reset the interface in the guest operating system to get the latest MAC address from the virtual device and acquire an IP address.