NSX-T Routing for a Multi-Region SDDC for the Management Domain

Design the routing configuration in NSX-T Data Center for multiple regions to support network span between regions for management applications that require resilient connectivity at multiple locations and to enable granular control of traffic from and to each region.

North-South Routing in Multiple Regions

In a routing design for a multi-region deployment, you identify which regions an SDN network must span and which regions must let ingress and egress traffic.

Network traffic that is entering or leaving the SDN networks with region preference and failover is a key design choice for a multi-site deployment. This design does not use local-egress, that is, traffic leaving and entering any location which the network spans. Instead,this design uses a preferred and failover region for all networks. The complexities of local-egress that is, controlling local-ingress to prevent asymmetrical routing, is not necessary for this design.

In this design, an NSX-T component can be primary at one or more regions. During a region failure, setting a network component as primary at another region is a manual operation.

Figure 1. North-South Routing in a Multi-Region SDDC

Tier-0 Gateways

In NSX-T Federation, a Tier-0 gateway can span multiple regions. Each region contains a logical unit of the Tier-0 gateway which is assigned to the edge cluster in the region and is configured to interface with the data center top of rack switches in the region.

Each region that is in the scope of a Tier-0 gateway can be configured as primary or secondary. Primary regions pass traffic for any other SDN service such as Tier-0 logical segments or Tier-1 gateways. Secondary regions route traffic locally but do not egress traffic outside the SDN or advertise networks in the data center.

When deploying an additional region, the Tier-0 gateway in the first region is extended to the new region.

In this design, the Tier-0 gateway in each region is configured as primary. Although the Tier-1 gateway technically supports local-egress, the design does not recommend the use of local-egress. Ingress and egress traffic is controlled at the Tier-1 gateway level.

Table 1. Design Decisions on the Tier-0 Gateway Configuration for a Multi-Region SDDC
Decision ID	Design Decision	Design Justification	Design Implication
SDDC-MGMT-VI-SDN-076	For a dual-region SDDC, extend the management domain active-active Tier-0 gateway to the second region.	Supports ECMP north-south routing on all nodes in the NSX-T Edge cluster. Enables support for cross-region Tier-1 gateways and cross-region network segments.	Active-active Tier-0 gateways cannot provide stateful services such as NAT.
SDDC-MGMT-VI-SDN-077	For a dual-region SDDC, set the Tier-0 gateway as primary in all regions.	In NSX-T Federation, a Tier-0 gateway lets egress traffic from connected Tier-1 gateways only in its primary region. Local ingress and egress traffic is controlled independently at the Tier-1 level. No segments are provisioned directly to the Tier-0 gateway. In a workload domain, this architecture improves flexibility for unique use cases. A mixture of network spans (isolated to a region or spanning multiple regions) is enabled without requiring additional Tier-0 gateways and hence edge nodes. If a region failure occurs, the region-specific networking in the other regions will remain available without manual intervention.	None.

Each region has its own NSX-T Edge cluster with associated uplink VLANs for north-south traffic flow for that data center. Similarly to the single-region design, each Tier-0 gateway unit peers with the top of rack switches over eBGP.

The NSX-T Tier-0 gateway behaves like a standard eBGP router. By default, any routes that the Tier-0 gateway learns from one eBGP neighbor are advertised to the other eBGP neighbours. Because the underlying network connectivity between the regions is not an independent path, but rather relies on the data center networks for connectivity, avoid advertising any learned networks from one data center to another. To prevent route advertising, apply the no-export BGP community to any routes learned from the top of rack switches in each data center.

Figure 2. BGP Peering to Top of Rack Switches

Table 2. Design Decisions on Routing Configuration for a Multi-Region SDDC
Decision ID	Design Decision	Design Justification	Design Implication
SDDC-MGMT-VI-SDN-078	For a dual-region SDDC, from the global Tier-0 gateway, establish BGP neighbor peering to the top of rack switches in the second region.	Enables the learning and advertising of routes between in the second region. Facilitates the automated failover of networks from the first to the second region.	None.
SDDC-MGMT-VI-SDN-079	For a dual-region SDCC, on the global Tier-0 gateway, apply the no-export BGP community to all routes learned from external neighbors.	You disable re-advertising data center routes that are learned from the first-region data center networks to the second-region data center or the opposite. By default, routes learned from one autonomous system over eBGP will be advertised to another autonomous system as a valid path connected over the NSX-T SDN. Because the NSX-T SDN in the first and second regions does not have an independent path between those autonomous systems, re-advertising data center networks would give a false indication of a valid, independent path.	None.

Tier-1 Gateways

A Tier-1 gateway can span one or more regions. Similarly to a Tier-0 gateway, you can configure a region as primary or secondary for a Tier-1 gateway. The gateway passes ingress and egress traffic for the logical segments connected to it.

Any logical segments connected to the Tier-1 gateway follow the span of the Tier-1 gateway. If the Tier-1 gateway spans Region A and Region B, any segments connected to that gateway become available in both regions. To define which regions a Tier-1 gateway spans, you associate the Tier-1 gateway with the edge cluster at each region.

Using a Tier-1 gateway enables more granular control on logical segments in the primary and secondary regions. In this multi-region design, you use three Tier-1 gateways – one for Region A only segments, one for Region B only segments, and one for segments which span Region A and Region B.

Table 3. Region Configuration of the Tier-1 Gateways for Multiple Regions
Tier-1 Gateway	Region A	Region B	Ingress-Egress
Cross-Region	Primary	Secondary	Primary - Region A Failover - Region B
Region-A	Primary	-	Region A only
Region-B	-	Primary	Region B only

The Tier-1 gateway advertises its networks to the connected region-specific unit of the Tier-0 gateway. In the case of primary-secondary location configuration, the Tier-1 gateway advertises its networks only to the Tier-0 gateway unit in the region where the Tier-1 gateway is primary. The Tier-0 gateway unit then re-advertises those networks to the data center in the regions where that Tier-1 gateway is primary. During a region failover, the IT administratormust manually set the Tier-1 gateway in Region B as primary. Then, networks become advertised through Region B. The Tier-1 gateway does not advertise its attached networks through the secondary region.

Table 4. Design Decisions on Tier-1 Gateway Configuration for a Multi-Region SDDC
Decision ID	Design Decision	Design Justification	Design Implication
SDDC-MGMT-VI-SDN-080	For a dual-region SDDC, use Tier-1 gateways to control the span of networks and ingress and egress traffic in the primary region.	Enables a mixture of network spans (isolated to a region or spanning multiple regions) without requiring additional Tier-0 gateways and hence edge nodes.	To control region span, a Tier-1 gateway must be assigned to an edge cluster and hence has the Tier-1 SR component. East-west traffic between Tier-1 gateways with SRs need to physically traverse an edge node.
SDDC-MGMT-VI-SDN-081	For a dual-region SDDC, use a global cross-region Tier-1 gateway and connect it to the Tier-0 gateway for cross-region networks.	Enables network span between the regions because virtual network segments follow the span of the gateway they are attached to. Creates a two-tier routing architecture.
SDDC-MGMT-VI-SDN-082	For a dual-region SDDC, assign the NSX-T Edge cluster in each region to the global cross-region Tier-1 gateway. Set the first region as primary and the second region as secondary.	Enables cross-region network span between the first and second regions. Enables deterministic ingress and egress traffic for the cross-region network. If a region failure occurs, enables deterministic failover of the Tier-1 traffic flow. During the recovery of the primary region, enables deterministic failback of the Tier-1 traffic flow, preventing unintended asymmetrical routing. Eliminates the need to use BGP attributes in the primary and secondary regions to influence the region preference and failover.	You must manually fail over and fail back the cross-region network from the standby NSX-T Global Manager.
SDDC-MGMT-VI-SDN-083	For a dual-region SDDC, allocate a Tier-1 gateway in each region for region-specific networks and connect it to the cross-region Tier-0 gateway.	Creates a two-tier routing architecture. Enables site-specific networks that are not to span between region A and region B. Guarantees that site-specific networks to remain available in if a region failure occurs in another region.	None.
SDDC-MGMT-VI-SDN-084	For a dual-region SDDC, assign the NSX-T Edge cluster in the first region to the region-specific Tier-1 gateway in Region A, and the NSX-T Edge cluster in the second region to the region-specific Tier-1 gateway in Region B.	Enables region-specific networks to be isolated to their specific regions. Enables deterministic flow of ingress and egress traffic for the region-specific networks.	You can use the service router that is created for the Tier-1 gateway for networking services. However, such configuration is not required for network connectivity.
SDDC-MGMT-VI-SDN-085	For a dual-region SDDC, set each region-specific Tier-1 gateway only as primary in the home region. Avoid setting the gateway as secondary in the other region.	Prevents the need to use BGP attributes in primary and secondary regions to influence the region ingress-egress preference.	None.