VMware SD-WAN supports IPv6 addresses to configure the Edge Interfaces and Edge WAN Overlay settings.

The VCMP tunnel can be setup in the following environments: IPv4 only, IPv6 only, and dual stack.

Mixed Environment on Edge to Edge Network

If the initiator is dual-stack and the responder is single-stack, then the tunnel preference of initiator is ignored and tunnel is formed based on IP type of the responder. In other cases, the tunnel preference of the initiator takes precedence. You cannot establish overlay between an IPv4 only and IPv6 only Interfaces.

In the above example, the Edge B1 has dual stack Interface. The Edge B1 can build IPv4 VCMP to the IPv4 only Interface on Edge B2 (unpreferred tunnel) and IPv6 VCMP to the IPv6 only Interface on Edge B3 (preferred tunnel).

Mixed Environment on Edge to Gateway Network

When a dual-stack (both IPv4 and IPv6 activated) Edge connects to a single-stack Gateway (IPv4 only), IPv4 tunnel is established.

In the above illustration, the IPv4-only Gateway is connected to Edges E1 and E2 that have dual stack Interfaces with preference as IPv6. An IPv4 tunnel is established between the Gateway and Edges.

In this scenario, the Edges do not learn the public IPv6 endpoints of the other Edges/Hubs from the Gateway, as the Gateway is not IPv6 capable. They only learn the IPv4 endpoints, along with the information that the overlay preference of the other Edge or Hub is IPv6. Even though both the devices negotiate and understand that their overlay preference matches (IPv6), they will not be able to form IPv6 tunnels between them due to lack of IPv6 endpoint information. In addition, the overlay preference negotiation match (both IPv6) prevents the devices from forming IPv4 tunnels with each other.

In such cases where an Edge is connected to an IPv4-only Gateway, it is recommended to set the overlay preference as IPv4 so that the Edges can establish IPv4 tunnels among themselves.

Note: It is recommended not to include IPv4-only Gateway into a Gateway Pool with dual stack Gateways.

Dual Stack Environment

When all the Edges and Gateways are on dual stack, the tunnel preference is selected as follows:

  • Edge to Gateway – The initiator, Edge, always chooses the tunnel type based on the tunnel preference.
  • Edge to Hub – The initiator, Spoke Edge, always chooses the tunnel type based on the tunnel preference.
  • Dynamic Branch to Branch – When there is a mismatch in the tunnel preference, the connection uses IPv4 addresses to ensure consistent and predictable behavior.

For Edge to Edge connections, the preference is chosen as follows:

  • When the Interfaces of Edge peers are set with same preference, the preferred address type is used.
  • When the Interfaces of Edge peers are set with different preferences, then the preference of the initiator is used.
Note: When both the ends are on dual stack, with IPv4 as the preference and the overlay established with IPv4, the IPv6 overlay will not be established.

In the above Illustration, all the Edges are on dual stack with the following preferences:

  • Edge B1: IPv6
  • Edge B2: IPv6
  • Edge B3: IPv4

In the above example, a dynamic Edge to Edge tunnel is built over IPv4 between the Edges B2 and B3, regardless of the site that initiates the connection.

Impact of IPv6 Tunnel on MTU

When a branch has at least one IPv6 tunnel, DMPO uses this tunnel seamlessly along with other IPv4 tunnels. The packets for any specific flow can take any tunnel, IPv4 or IPv6, based on the real time health of the tunnel. An example for specific flow is path selection score for load balanced traffic. In such cases, the increased size for IPv6 header (additional 20 bytes) should be taken into account and as a result, the effective path MTU will be less by 20 bytes. In addition, this reduced effective MTU will be propagated to the other remote branches through Gateway so that the incoming routes into this local branch from other remote branches reflect the reduced MTU.

When there are single or multiple sub Interfaces available, the Route Advertisement MTU is not updated properly in sub Interface. The sub Interfaces inherit the MTU value from the Parent Interface. The MTU values received on sub interfaces are ignored and only the parent interface MTU is honored. When an Edge has single sub Interface or multiple sub Interfaces, you must turn off the MTU option in the Route Advertisement of the peer Router. As an alternative, you can modify the MTU value of a sub Interface in a user-defined WAN overlay. For more information, see Configure Edge WAN Overlay Settings.

IPv6 Capability of Edge

The IPv6 Capability of an Edge is decided based on the IPv6 admin status of any interface. The Edge should have any one of the following activated with IPv6: Switched-VLAN, Routed-Interface, Sub-Interface, Loopback-Interface. This allows to categorize the Edge as IPv6 capable node to receive the IPv6 remote routes from Gateway.

Note:

Hubs always receive IPv6 remote routes, irrespective of their IPv6 Capability.

Limitations of IPv6 Address Configuration
  • SD-WAN Edge does not support configuring private overlay on one address family and public overlay on the other address family in the same routed Interface. If configured, the SD-WAN Edge would initiate the tunnel using the preferred address family configured on the routed Interface.
  • The tunnel preference change can be disruptive for the PMTU overhead. When there is a change in the configuration to setup all Interfaces with IPv4 tunnel preference, the Edge to Edge or Hub to Spoke tunnels may be torn down and re-established to use the IPv4 overhead to ensure that the tunnel bandwidth is used optimally.
  • In an Interface with different IP links, the bandwidth measured by the preferred tunnel or link is inherited by other links. Whenever the tunnel preference is changed for a link from IPv6 to IPv4 or vice versa, the link bandwidth is not measured again.
  • When there is a change in the tunnel address or change in the preference of the tunnel from IPv6 to IPv4 address or vice versa, the existing flows are dropped in a Hub or Spoke. You should flush the flows in the Hub or Spoke to recover the bi-directional traffic.
  • While monitoring the events for a Gateway in Operator Events page or an Edge in the Monitor > Events page, when the Gateway or Edge is not able to send heartbeat, the corresponding event message displays the IPv6 address with hyphens instead of colons, in the following format: x-x-x-x-x-x-x-x. This does not have any impact on the functionality.
  • Edge version running 4.x switched interface does not support IPv6 address.
  • SD-WAN Edge does not use new IPv6 prefixes if it has multiple IPv6 prefixes because it might cause tunnel flaps. In this case, Edge prioritizes the old IPv6 prefix. If there is a need to use the new IPv6 prefix, it is recommended to bounce the Internet-facing WAN interface or restart the Edge for immediate recovery. Alternatively, you can wait until the old address entry ages out.

Management Traffic and IP Addresses

When Edge goes offline with multiple combination of IP address family being used, the Edge will not be able to communicate with the Orchestrator. This happens when sending direct traffic and link selection fails.

In Dual stack Orchestrator and Edge, the Management Plane Daemon (MGD) always prefers IPv6 address for MGD to Orchestrator communication. If IPv6 fails, then it falls back to IPv4. The following matrix shows IP family chosen by MGD for Orchestrator communication.

Orchestrator
Edge IPv4 IPv6 Dual
IPv4 MGD traffic is IPv4 Mis-matched family MGD traffic is IPv4
IPv6 Mis-matched family MGD traffic is IPv6 MGD traffic is IPv6
Dual MGD traffic is IPv4 MGD traffic is IPv6 MGD traffic is IPv6

MGD traffic is always sent over overlay through cloud Gateway unless all the paths to Gateway are down. In this case MGD traffic to Orchestrator is sent directly. The following is the logic to drain packet direct.

  1. Loop over all the Interface. In the following cases, the Edge is left with Interfaces consisting of activated WAN links only.
    1. Interface on which WAN overlay is deactivated is not considered.
    2. When Interface is single stack with IPv6 and traffic is IPv4, then it is not considered.
    3. When Interface is single stack with IPv4 and traffic is IPv6, then it is not considered.
  2. Loop over WAN link on Interface. In the following cases, the Edge is left with a WAN link that could be used even if paths are down to cloud Gateway.
    1. If WAN link is Standby, then it is not considered.
    2. If WAN link is Private, then it is not considered.

You can configure IPv6 addresses for the following: