When you create a transport zone or a logical switch, you must select a replication mode. Understanding the different modes can help you decide which is most appropriate for your environment.
Each ESXi host prepared for NSX is configured with a VXLAN tunnel endpoint (VTEP). Each VXLAN tunnel endpoint has an IP address. These IP addresses can be in the same subnet or in different subnets.
- Unicast Replication Mode
- Multicast Replication Mode
- Hybrid Replication Mode
Summary of Replication Modes
|Replication Mode||Method of BUM Replication to VTEPs on the Same Subnet||Method of BUM Replication to VTEPs on a Different Subnet||Physical Network Requirements|
|Multicast||Layer 2 multicast||Layer 3 multicast||
|Hybrid||Layer 2 multicast||Unicast||
Unicast Replication Mode
Unicast replication mode does not require the physical network to support layer 2 or layer 3 multicast to handle the BUM traffic within a logical switch. Using unicast mode completely decouples logical networks from the physical network. Unicast mode replicates all the BUM traffic locally on the source host and forwards the BUM traffic in a unicast packet to the remote hosts. In unicast mode, you can have all VTEPs in one subnet, or in multiple subnets.
One subnet scenario: If all host VTEP interfaces belong to a single subnet, the source VTEP forwards the BUM traffic to all remote VTEPs. This is known as head-end replication. Head-end replication might result in unwanted host overhead and higher bandwidth usage. The impact depends on the amount BUM traffic and the number of hosts and VTEPs within the subnet.
Multiple subnet scenario: If the host VTEP interfaces are grouped into multiple IP subnets, the source host handles the BUM traffic in two parts. The source VTEP forwards the BUM traffic to each VTEP in the same subnet (the same as the one subnet scenario). For VTEPs in remote subnets, the source VTEP forwards the BUM traffic to one host in each remote VTEP subnet and sets the replication bit to mark this packet for local replication. When a host in the remote subnet receives this packet and finds the replication bit set, it sends the packet to all the other VTEPs in its subnet where the logical switch exists.
Therefore, unicast replication mode scales well in network architectures with many VTEP IP subnets as the load is distributed among multiple hosts.
Multicast Replication Mode
Multicast replication mode requires that both layer 3 and layer 2 multicast is enabled in the physical infrastructure. To configure multicast mode, the network administrator associates each logical switch with an IP multicast group. For ESXi hosts that are hosting VMs on a specific logical switch, the associated VTEPs join the multicast group using IGMP. The routers track the IGMP joins and create a multicast distribution tree between them using a multicast routing protocol.
When hosts replicate BUM traffic to VTEPs in the same IP subnet, they use layer 2 multicast. When hosts replicate BUM traffic to VTEPs in different IP subnets, they use layer 3 multicast. In both cases, the replication of BUM traffic to remote VTEPs is handled by the physical infrastructure.
Even though IP multicast is a well-known technology, the deployment of IP multicast in the data center is often considered a roadblock for different technical, operational, or administrative reasons. The network administrator must be careful about the maximum supported multicast states in the physical infrastructure to enable the one-to-one mapping between the logical switch and the multicast group. One of the benefits of virtualization is that it allows scaling the virtual infrastructure without exposing additional states to the physical infrastructure. Mapping logical switches to "physical" multicast groups breaks this model.
Hybrid Replication Mode
Hybrid mode is a hybrid between unicast and multicast replication modes. In hybrid replication mode, host VTEPs use layer 2 multicast to distribute BUM traffic to peer VTEPs in the same subnet. When host VTEPs replicate BUM traffic to VTEPs in different subnets, they forward the traffic as unicast packets to one host per VTEP subnet. This receiving host in turn uses layer 2 multicast to send the packets to other VTEPs in its subnet.
Layer 2 multicast is more common in customer networks than Layer 3 multicast as it is typically easy to deploy. The replication to different VTEPs in the same subnet is handled in the physical network. Hybrid replication can be a significant relief for the source host for BUM traffic if there are many peer VTEPs in the same subnet. With hybrid replication, you can scale up a dense environment with little or no segmentation.