In an environment with multiple availability zones, Layer 2 networks must be stretched between the availability zones by the physical infrastructure. You also must provide a Layer 3 gateway that is highly available between availability zones. The method for stretching these Layer 2 networks and providing a highly available Layer 3 gateway is vendor-specific.

VLANs and Subnets for Multiple Available Zones

This section displays a sample configuration for an environment with multiple availability zones. The VM management, Uplink 01, Uplink 02, and Edge overlay networks in each availability zone must be stretched to facilitate failover of the NSX Edge appliances between availability zones. The Layer 3 gateway for the management and Edge overlay networks must be highly available across the availability zones.

Note: If VLAN is stretched between AZ1 and AZ2, the Layer 3 network must also be stretched between the two AZs.
Table 1. Stretched Cluster Subnet Requirements

Function

Availability Zone 1

Availability Zone 2

HA Layer 3 Gateway

Recommended MTU
VM Management VLAN ✓ ✓ ✓ 1500

Management VLAN (AZ1)

✓

X

✓

1500

vMotion VLAN

✓

X

✓

9000

vSAN VLAN (AZ1)

✓

X

✓

9000

NSX Host Overlay VLAN

✓

X

✓

9000

NSX Edge Uplink01 VLAN

✓

✓

X

9000

NSX Edge Uplink02 VLAN

✓

✓

X

9000

NSX Edge Overlay VLAN

✓

✓

✓

9000

Management VLAN (AZ2)

X

✓

✓

1500

vMotion VLAN (AZ2)

X

✓

✓

9000

vSAN VLAN (AZ2)

X

✓

✓

9000

NSX Host Overlay VLAN (AZ2)

X

✓

✓

9000

Networking for Multiple Availability Zones

There are specific physical data center network requirements for a topology with multiple availability zones. For information about the vSAN witness appliance requirements, see vSAN Witness Network Design in the VMware Cloud Foundation Design Guide.

Table 2. Physical Network Requirements for Multiple Availability Zone

Component

Requirement

MTU

 VLANs which are stretched between availability zones must meet the same requirements as the VLANs for intra-zone connection including MTU. MTU value must be consistent end-to-end including components on the inter-zone networking path. Set MTU values as follows.
  • MTU for all VLANs and Switch Virtual Interfaces (vMotion, Geneve, and Storage) to jumbo frames.
  • Management MTU to 1500.
  • Geneve overlay requires a minimum MTU of 1600.

Layer 3 gateway availability

For VLANs that are are stretched between available zones, configure data center provided method to failover the Layer 3 gateway between availability zones. For example, VRRP or HSRP.

DHCP availability

For VLANs that are stretched between availability zones, provide high availability for the DHCP server so that a failover operation of a single availability zone will not impact DHCP availability.

BGP routing

Each availability zone data center must have its own Autonomous System Number (ASN).

Ingress and egress traffic

  • For VLANs that are stretched between availability zones, traffic flows in and out of a single zone. Local egress is not supported.

  • For VLANs that are not stretched between availability zones, traffic flows in and out of the zone where the VLAN is located.

  • For NSX virtual network segments that are stretched between regions, trafficflows in and out of a single availability zone. Local egress is not supported.

Latency

 vSphere
  • Less than 150 ms latency RTT for vCenter Server connectivity.
  • Less than 150 ms latency RTT for vMotion connectivity.
  • Less than 5 ms latency RTT for VSAN hosts connectivity.

vSAN

  • Less than 200 ms latency RTT for up to 10 hosts per site.
  • Less than 100 ms latency RTT for 11-15 hosts per site.

NSX Managers

  • Less than 10 ms latency RTT between NSX Managers
  • Less than 150 ms latency RTT between NSX Managers and transport nodes.