You can apply Equal-Cost Multi-Path (ECMP) routing on ESXi hosts with Enhanced Network Stack (ENS) and Standard switch.

ECMP has two modes L3 ECMP and L4 ECMP. The support for L4 ECMP has started from NSX 4.2.1. You can apply L3 ECMP to only a Standard switch, but you can apply both L3 ECMP and L4 ECMP to an ENS switch.

The L4 ECMP mode is only supported for VDS in the following two modes:

  1. Enhanced Data Mode (Interrupt)
  2. Enhanced Data Mode (Polling)

You can apply the ECMP mode either directly to a Transport node or through the Transport Node Profile. Note that the ECMP mode configuration is not supported in the sub-TNP level. The TNP level ECMP mode configuration is inherited in the Sub-TNP profile.

To apply the ECMP mode to a Transport node, run the following API.

PUT https://<nsx-mgr>/policy/api/v1/infra/sites/<site-id>/enforcement-points/<enforcementpoint-id>/host-transport-nodes/

To apply the ECMP mode through a Transport Node Profile, run the following API.

PUT https://<nsx-mgr>/policy/api/v1/infra/host-transport-node-profiles/<transport-node-profile-id>

For more details about the APIs, see the NSX API Guide.

The ECMP 5-tuple feature (hashing on protocol number, source and destination address, and source and destination port) is disabled by default in ESXi. You can enable it by setting the parameter lb_ecmp to true with the following API call. 
PUT https://<nsx-manager>/policy/api/v1/infra/connectivity-global-config
{
    "lb_ecmp": true,
    ...
}

You can see the current lb_ecmp value with the API GET https://<nsx-manager>/policy/api/v1/infra/connectivity-global-config.

When upgrading from NSX 4.2 to NSX 4.2.1, the lb_ecmp configuration is realized based on the switch type. Once the transition to NSX 4.2.1 is complete, changing the lb_ecmp parameter will not have any affect on the overall NSX deployment.

Notes about 5-tuple ECMP:
  • When 5-tuple ECMP for ESXi is enabled, it will be applied to all ESXi hosts. Hosts that do not have ENS enabled might show unexpected behavior.
  • Do not enable this feature if you have non-ENS hosts, or a mix of ENS and non-ENS hosts.
  • There are appropriate validations to prevent you from enabling L4 ECMP for non-ENS switches.
  • When 5-tuple ECMP is enabled, ENS-enabled hosts will consume significantly more flows. When the flow limit is reached, the performance of the host might be impacted.
  • Certain topologies, such as a load balancer, where layer 3 does not provide enough path diversity, might benefit more with 5-tuple ECMP enabled.