Support for Jumbo Frames in Avi Load Balancer

This section explains the support for jumbo frames in Avi Load Balancer.

Avi Load Balancer supports MTU greater than 1500 on the Service Engine interface. Ensure that the larger MTU is supported throughout the network.

Configuring Path MTU Discovery

PMTU discovery is supported in Service Engines. You can enable or deactivate path MTU discovery (PMTUD) for IPv4 and IPv6 under SE group using path_mtu_discovery_v4 and path_mtu_discovery_v6 knobs. By default, path_mtu_discovery_v4 is set to False and path_mtu_discovery_v6 is set to True.

| path_mtu_discovery_v4 		| False  	        |
| path_mtu_discovery_v6 		| True 			|

Configuring SE MTU

You can manually configure to allow jumbo frames in the datapath using Service Engine property se_mtu. After configuring manually, the range will be a user-configured value between 512-9000 and will not have any default value.

The global MTU is an SE property used to limit the MTU if there are some encapsulation overheads that can enlarge the packet over the original MTU. You can replace the global MTU with se_mtu for the same purpose.

The se_mtu always overrides global MTU if configured. For instance, if you configure se_mtu to 9K, the system does not depend on the global MTU value. Global MTU is kept only for backward compatibility, i.e., if you configure global MTU in an earlier release and do an upgrade, then the global MTU should still take effect unless you configure se_mtu later.

You can configure se_mtu with jumbo size on the serviceenginegroup. This will change the MTU to the value for all the VNICs on all the SEs in the Service Engine group. If it is not set, then it will be backward compatible with global_mtu of Service Engine properties.

You need to set all the VNIC’s MTU to the global_mtu property if it has been configured or to VNIC’s original MTU if it is not configured.

Note:

The se_mtu affects the MTU of the SE’s interfaces only.

Ensure that the jumbo MTU is configured on all the nodes along the data path for the end-to-end jumbo frame traffic to work. The configuration should be accurate for the results to be as expected.

Configuration Example

[admin:10-79-109-168]: > configure serviceenginegroup Default-Group

[admin:10-79-109-168]: serviceenginegroup> se_mtu 9000

[admin:10-79-109-168]: serviceenginegroup> save

You can configure MTU through global_mtu or se_mtu, where global_mtu can be configured through serviceengineproperties and se_mtu can be configured through serviceenginegroup. The se_mtu takes precedence in case both these fields are set.

To verify if the configuration is successfully applied, use show serviceengine <se-name> interface command and check vnic_mtu for a specific vnic.

[admin:vmwareft-ctlr1]: > show serviceengine Avi-se-wmwph interface | egrep  "vnic_name|vnic_mtu"

|   vnic_name                              | eth8                         |
|   vnic_mtu                               | 9000                         |
|   vnic_name                              | eth1                         |
|   vnic_mtu                               | 9000                         |

Note:

You cannot update the MTU using ifconfig code in Avi Load Balancer namespace. You can only update the MTU either by using se_mtu (SE group property) or global_mtu (SE properties).

Jumbo Frames - Interface Families Supported


	Jumbo Frame Support	Jumbo Frame Support on VLAN Interface
Avi Load Balancer supported Intel NICs	Yes	Yes
Avi Load Balancer supported Mellanox NICs	Yes	Yes
Avi Load Balancer supported Broadcom NICs	Yes	Yes
VMXNET3 interfaces	Yes	No
VIRTIO interfaces	Yes	Yes
Avi Load Balancer non-DPDK mode of operation	Yes	Yes

KNI MTU cannot exceed 1500 even when NIC MTU is configured.
se_mtu vs global_mtu — global_mtu is an SE property that is used to configure the interface MTU. This can be used to accommodate any encapsulation overhead that can enlarge the packet beyond the 1500 MTU.
You can replace global_mtu with se_mtu as se_mtu supports the jumbo frame. The se_mtu configuration parameter/ field always overrides global_mtu if configured. For instance, if you configure se_mtu to 9000, the system does not depend on the global_mtu value.

Note:

The global_mtu is retained only for backward compatibility, that is, if you configure global_mtu in an earlier release and do an upgrade, then global_mtu must still take effect unless you configure se_mtu later.

For information on the list of supported NICs, see System Requirements section in Preparing the Installation in VMware Avi Load BalancerInstallation Guide.