In the default configuration, your SDDC network has a single edge (T0) router through which all North-South traffic flows. This edge supports the default traffic group, which is not configurable. If you need additional bandwidth for the subset of this traffic routed to SDDC group members, a Direct Connect Gateway attached to an SDDC group, VMware HCX Service Mesh, or to the Connected VPC, you can reconfigure your SDDC to be Multi-Edge by creating traffic groups, each of which creates an additional T0 router.
A traffic group uses an association map to associate a prefix list of CIDR blocks to one of the T0 gateways that support non-default traffic groups in your SDDC. Prefixes from custom T1 gateways are not valid in traffic groups. Prefix lists are independent of gateways and do not use standard destination-based routing. Instead, they use source-based forwarding. Traffic from an IP matching a prefix list is forwarded to the T0 edge mapped to the prefix list's associated traffic group for outbound routing. If that T0 does not support the destination (see Note) then the IP is forwarded on to the default T0 for routing. This adds an additional hop, so try to make sure that the traffic group's associated T0 supports traffic being sent from a prefix-list IP.
VPN traffic, as well as DX traffic to a private VIF must pass through on the default T0 and cannot be routed to a non-default traffic group. In addition, because NAT rules always run on the default T0 router, additional T0 routers cannot handle traffic affected by SNAT or DNAT rules. This includes traffic to and from the SDDC's native Internet connection. It also includes traffic to the Amazon S3 service, which uses a NAT rule and must go through the default T0. Keep these limitations in mind when you create prefix lists.
Prerequisites
-
Before you can create traffic groups, you must use VMware Transit Connect™ to connect your SDDC to a VMware Managed Transit Gateway (VTGW). See Creating and Managing SDDC Deployment Groups.
-
Traffic groups can be created only in SDDCs that have large-size management appliances and at least four hosts. See Upsize SDDC Management Appliances in the VMware Cloud on AWS Operations Guide for information about changing an SDDC's management appliance size from medium to large. See Add Hosts for information about adding hosts to an SDDC.
-
Each traffic group deploys two Edge VMs in addition to the two default Edge VMs. Because Edge VMs cannot share the same host and meet performance requirements, you'll need at least two hosts per traffic group and an additional two hosts for the default traffic group in the management cluster (Cluster-1). The number of traffic groups that an SDDC can support depends on the number of management hosts, and can be represented with a formula like this:
TG=(mgmt-hosts - 2)/2|MAX
where TG represents the maximum number of traffic groups that the SDDC can support and mgmt-hosts is the number of hosts the SDDC management cluster. Regardless of the calculated value of TG, SDDC traffic group support is capped at the Maximum number of Multi-Edge SDDC Traffic Groups per SDDC shown in VMware Configuration Maximums ( MAX).
Procedure
Example: Route Table Changes After Adding a Traffic Group
This simplified example shows the effect of creating traffic group and associating it with a prefix list of just two host routes (/32).
- Initial configuration
-
Assume these values for route table entries in the default traffic group and the Compute Gateway (CGW) before adding the first traffic group (which creates an additional T0 router).
Table 1. Default Routes Subnet Next Hop 0.0.0.0/0 Internet Gateway 192.168.150.51/24 CGW 192.168.151.0/24 CGW VTGW, DXGW subnets VTGW, DXGW connections Management CIDR MGW Table 2. CGW Routes With the Default Traffic Group Subnet Next Hop 0.0.0.0/0 Default T0 192.168.150.0/24 Default T0 192.168.151.0/24 Default T0 - Multi-Edge configuration
-
After the first traffic group is created, new routes are added on the default T0. Assuming that the prefix list associated with the traffic group has these entries:
192.168.150.100/32 192.168.151.51/32
then the route tables for the default T0, new T0, and CGW end up like this.Table 3. Default T0 Routes After Adding a Traffic Group Subnet Next Hop 0.0.0.0/0 Internet Gateway 192.168.150.0/24 CGW 192.168.150.100/32 New T0 192.168.151.0/24 CGW 192.168.151.51/32 New T0 VTGW, DXGW subnets VTGW, DXGW connections Management CIDR MGW Table 4. Routes on the New Traffic Group Subnet Next Hop 0.0.0.0/0 Default T0 192.168.150.100/32 CGW 192.168.151.51/32 CGW VTGW, DXGW subnets VTGW, DXGW connections Management CIDR MGW Table 5. CGW Routes With an Additional Traffic Group Subnet Next Hop 0.0.0.0/0 Default T0 192.168.150.0/24 Default T0 192.168.150.100/32 New T0 192.168.151.0/24 Default T0 192.168.151.51/32 New T0