Enable SR-IOV on KVM

To enable the SR-IOV mode on KVM, perform the following steps.

Prerequisites

This requires a specific NIC card. The following chipsets are certified by VMware to work with the SD-WAN Gateway and SD-WAN Edge.

Intel 82599/82599ES
Intel X710/XL710

Note: Before using the Intel X710/XL710 cards in SR-IOV mode on KVM, make sure the supported Firmware and Driver versions specified in the Deployment Prerequisites section are installed correctly.

Note: SR-IOV mode is not supported if the KVM Virtual Edge is deployed with a High-Availability topology. For High-Availability deployments, ensure that SR-IOV is not enabled for that KVM Edge pair.

To enable SR-IOV on KVM:

Enable SR-IOV in BIOS. This will be dependent on your BIOS. Login to the BIOS console and look for SR-IOV Support/DMA. You can verify support on the prompt by checking that Intel has the correct CPU flag.
```
cat /proc/cpuinfo | grep vmx
```

Add the options on Bboot (in /etc/default/grub).

GRUB_CMDLINE_LINUX="intel_iommu=on"

Run the following commands: update-grub and update-initramfs -u.
Reboot

Make sure iommu is enabled.

velocloud@KVMperf3:~$ dmesg | grep -i IOMMU
 [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.13.0-107-generic root=/dev/mapper/qa--multiboot--002--vg-root ro intel_iommu=on splash quiet vt.handoff=7 
 [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.13.0-107-generic root=/dev/mapper/qa--multiboot--002--vg-root ro intel_iommu=on splash quiet vt.handoff=7 
 [ 0.000000] Intel-IOMMU: enabled
 ….
 velocloud@KVMperf3:~$

Based on the NIC chipset used, add a driver as follows:
- For the Intel 82599/82599ES cards in SR-IOV mode:
  1. Download and install ixgbe driver from the Intel website.
  2. Configure ixgbe config (tar and sudo make install).
```
velocloud@KVMperf1:~$ cat /etc/modprobe.d/ixgbe.conf
```
  3. If the ixgbe config file does not exist, you have to create the file as follows.
```
options ixgbe max_vfs=32,32
options ixgbe allow_unsupported_sfp=1
options ixgbe MDD=0,0
blacklist ixgbevf
```
  4. Run the update-initramfs -u command and reboot the Server.
  5. Use the modinfo command to verify if the installation is successful.
```
velocloud@KVMperf1:~$ modinfo ixgbe and ip link
 filename: /lib/modules/4.4.0-62-generic/updates/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
 version: 5.0.4
 license: GPL
 description: Intel(R) 10GbE PCI Express Linux Network Driver
 author: Intel Corporation, <[email protected]>
 srcversion: BA7E024DFE57A92C4F1DC93
```
- For the Intel X710/XL710 cards in SR-IOV mode:
  1. Download and install i40e driver from the Intel website.
  2. Create the Virtual Functions (VFs).
```
echo 4 > /sys/class/net/device name/device/sriov_numvfs
```
  3. To make the VFs persistent after a reboot, add the command from the previous step to the "/etc/rc.d/rc.local" file.
  4. Deny list the VF driver
```
echo “blacklist i40evf” >> /etc/modprobe.d/blacklist.conf
```
  5. Run the update-initramfs -u command and reboot the Server.

Validating SR-IOV (Optional)

You can quickly verify if your host machine has SR-IOV enabled by using the following command:

lspci | grep -i Ethernet

Verify if you have Virtual Functions:

01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function(rev 01)