In the default configuration, your SDDC network has a single edge (T0) router through which all North-South traffic flows. This edge supports the default traffic group, which is not configurable. If you need additional bandwidth for the subset of this traffic routed to SDDC group members, a Direct Connect Gateway attached to an SDDC group, VMware HCX Service Mesh, or to the Connected VPC, you can reconfigure your SDDC to be Multi-Edge by creating traffic groups, each of which creates an additional T0 router.
A traffic group uses an association map to associate a prefix list of CIDR blocks to one of the T0 gateways that support non-default traffic groups in your SDDC. Prefixes from custom T1 gateways are not valid in traffic groups. Prefix lists are independent of gateways and do not use standard destination-based routing. Instead, they use source-based forwarding. Traffic from an IP matching a prefix list is forwarded to the T0 edge mapped to the prefix list's associated traffic group for outbound routing. If that T0 does not support the destination (see Note) then the IP is forwarded on to the default T0 for routing. This adds an additional hop, so try to make sure that the traffic group's associated T0 supports traffic being sent from a prefix-list IP.
VPN traffic, as well as DX traffic to a private VIF must pass through on the default T0 and cannot be routed to a non-default traffic group. In addition, because NAT rules always run on the default T0 router, additional T0 routers cannot handle traffic affected by SNAT or DNAT rules. This includes traffic to and from the SDDC's native Internet connection. It also includes traffic to the Amazon S3 service, which uses a NAT rule and must go through the default T0. Keep these limitations in mind when you create prefix lists.
Prerequisites
-
Before you can create traffic groups, you must use VMware Transit Connect™ to connect your SDDC to a VMware Managed Transit Gateway (VTGW). See Creating and Managing SDDC Deployment Groups.
-
Traffic groups can be created only in SDDCs that have large-size management appliances and at least four hosts. See Upsize SDDC Management Appliances in the VMware Cloud on AWS Operations Guide for information about changing an SDDC's management appliance size from medium to large. See Add Hosts for information about adding hosts to an SDDC.
-
Each traffic group deploys two Edge VMs in addition to the two default Edge VMs. Because Edge VMs cannot share the same host and meet performance requirements, you'll need at least two hosts per traffic group and an additional two hosts for the default traffic group in the management cluster (Cluster-1). The number of traffic groups that an SDDC can support depends on the number of management hosts, and can be represented with a formula like this:
TG=(mgmt-hosts - 2)/2|MAX
where TG represents the maximum number of traffic groups that the SDDC can support and mgmt-hosts is the number of hosts the SDDC management cluster. Regardless of the calculated value of TG, SDDC traffic group support is capped at the Maximum number of Multi-Edge SDDC Traffic Groups per SDDC shown in VMware Configuration Maximums ( MAX).
Procedure
- Create a prefix list.
Because Multi-Edge SDDCs use source-based routing in their traffic groups, prefix lists must contain source addresses, not destination addresses.
- On the IP Prefix List tab of the Traffic Groups page, click ADD IP PREFIX LIST and give the new prefix list a Name and optional Description.
- Click Set to display the Set Prefixes window, then click ADD PREFIX and fill in the CIDR block of an SDDC network segment that includes the source addresses of workload VMs whose traffic you want to include in the traffic group (and route over the additional edge).
Important: You cannot use the SDDC management CIDR block here or the CIDR block of a segment that provides the local IP address of a VPN. If you add any of these CIDRs to a prefix list, you won't be able to use the list in an association map.Click ADD to add the specified prefix to the list. To add prefixes or edit the ones already on the list, click
to open the prefixes editor.
- Click APPLY to apply your changes to the prefix list.
- When you're done adding or editing prefixes, click SAVE to save or create the prefix list.
- Associate a prefix list with a gateway. On the Traffic Groups tab of the Traffic Groups page, find the traffic group you want to work with, then click and select Edit.
Click the plus icon
in the
ASSOCIATION MAPS area, give the mapping a
Name and select an existing prefix list from the
Prefixes drop-down. Select a gateway from the
Gateway drop-down, and click
SAVE to create the association map.
- (Optional) To remove a traffic group, you must first remove its association maps.
- Find the traffic group on the Traffic Groups page. Click its button, then select Edit.
- Click the minus icon
to the right of the Status label under Association Maps to select the map for deletion, then click SAVE to delete the map. - Click CLOSE EDITING, then return to the traffic group on the Traffic Groups page. Click its ellipsis button and then select Delete.
It can take up to 30 minutes to remove a traffic group. Removing the traffic group removes the T0 router that was created to support it. HCX, if in use, creates its own association map, which you can view but not modify. To remove an association map created by HCX, you have to uninstall HCX. See Uninstalling VMware HCX in the VMware HCX User Guide. - Find the traffic group on the Traffic Groups page. Click its
Example: Route Table Changes After Adding a Traffic Group
This simplified example shows the effect of creating traffic group and associating it with a prefix list of just two host routes (/32).
- Initial configuration
-
Assume these values for route table entries in the default traffic group and the Compute Gateway (CGW) before adding the first traffic group (which creates an additional T0 router).
Table 1. Default Routes Subnet Next Hop 0.0.0.0/0 Internet Gateway 192.168.150.51/24 CGW 192.168.151.0/24 CGW VTGW, DXGW subnets VTGW, DXGW connections Management CIDR MGW Table 2. CGW Routes With the Default Traffic Group Subnet Next Hop 0.0.0.0/0 Default T0 192.168.150.0/24 Default T0 192.168.151.0/24 Default T0 - Multi-Edge configuration
-
After the first traffic group is created, new routes are added on the default T0. Assuming that the prefix list associated with the traffic group has these entries:
192.168.150.100/32 192.168.151.51/32
then the route tables for the default T0, new T0, and CGW end up like this.
The new routes (192.168.150.100/32 and 192.168.151.51/32 in the example tables) use the new T0 as their next-hop, and the new T0 uses longest-prefix matching to route that traffic to the CGW.Table 3. Default T0 Routes After Adding a Traffic Group Subnet Next Hop 0.0.0.0/0 Internet Gateway 192.168.150.0/24 CGW 192.168.150.100/32 New T0 192.168.151.0/24 CGW 192.168.151.51/32 New T0 VTGW, DXGW subnets VTGW, DXGW connections Management CIDR MGW
The CGW route table is updated to create the traffic group by specifying the new T0 router as the next hop for the new routes.Table 4. Routes on the New Traffic Group Subnet Next Hop 0.0.0.0/0 Default T0 192.168.150.100/32 CGW 192.168.151.51/32 CGW VTGW, DXGW subnets VTGW, DXGW connections Management CIDR MGW Table 5. CGW Routes With an Additional Traffic Group Subnet Next Hop 0.0.0.0/0 Default T0 192.168.150.0/24 Default T0 192.168.150.100/32 New T0 192.168.151.0/24 Default T0 192.168.151.51/32 New T0
