In the default configuration, your SDDC network has a single edge (T0) router through which all North-South traffic flows. This edge supports the default traffic group, which is not configurable. If you need additional bandwidth for the subset of this traffic routed to SDDC group members, a Direct Connect Gateway attached to an SDDC group, VMware HCX Service Mesh, or to the Connected VPC, you can reconfigure your SDDC to be Multi-Edge by creating traffic groups, each of which creates an additional T0 router.

A traffic group uses an association map to associate a prefix list of CIDR blocks to one of the T0 gateways that support non-default traffic groups in your SDDC. Prefix lists are independent of gateways and consist of source IP addresses. Traffic from those addresses is routed to the T0 edge that supports the associated traffic group. You can create and update prefix lists at any time, but you cannot remove a prefix list if it is included in an association map. Associating a prefix list with a traffic group routes all traffic from CIDR blocks in the list through the T0 router created for the group.

Note:

VPN traffic, as well as DX traffic to a private VIF must pass through on the default T0 and cannot be routed to a non-default traffic group. In addition, because NAT rules always run on the default T0 router, additional T0 routers cannot handle traffic affected by SNAT or DNAT rules. This includes traffic to and from the SDDC's native Internet connection. It also includes traffic to the Amazon S3 service, which uses a NAT rule and must go through the default T0. Keep these limitations in mind when you create prefix lists.

Prerequisites

  • Before you can create traffic groups, you must use VMware Transit Connect™ to connect your SDDC to a VMware Managed Transit Gateway (VTGW). See Creating and Managing SDDC Deployment Groups in the VMware Cloud on AWS Operations Guide.

  • Traffic groups can be created only in SDDCs that have large-size management appliances and at least four hosts. See Upsize SDDC Management Appliances for information about changing an SDDC's management appliance size from medium to large. See Add Hosts for information about adding hosts to an SDDC.

  • The number of traffic groups that a multi-AZ (stretched cluster) SDDC can support depends on the number of hosts that the SDDC provides in each region, and can be represented with a formula like this:
    TG=(hosts-per-region - 2)/2
    where TG represents the maximum number of traffic groups that the SDDC can support and hosts-per-region is the number of hosts the SDDC deploys in each of the regions it occupies.

Procedure

  1. Log in to VMware Cloud Services at https://vmc.vmware.com.
  2. Click Inventory > SDDCs, then pick an SDDC card and click VIEW DETAILS.
  3. Click OPEN NSX MANAGER.
    You can also use the VMC Console Networking & Security tab for this workflow. The Networking & Security tab combines NSX-T Networking tab features like VPN, NAT, and DHCP with Security tab features like firewalls.
  4. Create a traffic group. On the Traffic Groups tab of the Traffic Groups page, click ADD TRAFFIC GROUP and give the new traffic group a Name, then click SAVE to create the traffic group and an additional T0 router for it.
    The Status of the traffic group transitions to In Progress while the new T0 edge is being created. It can take up to 30 minutes for the process to complete. When it does, the Status of the traffic group transitions to Success and you can create an association map for it.
  5. Create a prefix list.
    Because Multi-Edge SDDCs use source-based routing in their traffic groups, prefix lists must contain source addresses, not destination addresses.
    1. On the IP Prefix List tab of the Traffic Groups page, click ADD IP PREFIX LIST and give the new prefix list a Name and optional Description.
    2. Click Set to display the Set Prefixes window, then click ADD PREFIX and fill in the CIDR block of an SDDC network segment that includes the source addresses of workload VMs whose traffic you want to include in the traffic group (and route over the additional edge).
      Important: You cannot use the SDDC management CIDR block here or the CIDR block of a segment that provides the local IP address of a VPN. If you add any of these CIDRs to a prefix list, you won't be able to use the list in an association map.
      Click ADD to add the specified prefix to the list. To add prefixes or edit the ones already on the list, click Actions menu to open the prefixes editor.
    3. Click APPLY to apply your changes to the prefix list.
    4. When you're done adding or editing prefixes, click SAVE to save or create the prefix list.
  6. Associate a prefix list with a gateway. On the Traffic Groups tab of the Traffic Groups page, find the traffic group you want to work with, then click Actions menu and select Edit.
    Click the plus icon in the ASSOCIATION MAPS area, give the mapping a Name and select an existing prefix list from the Prefixes drop-down. Select a gateway from the Gateway drop-down, and click SAVE to create the association map.
  7. (Optional) To remove a traffic group, you must first remove its association maps.
    1. Find the traffic group on the Traffic Groups page. Click its Actions menu button, then select Edit.
    2. Click the minus icon to the right of the Status label under Association Maps to select the map for deletion, then click SAVE to delete the map.
    3. Click CLOSE EDITING, then return to the traffic group on the Traffic Groups page. Click its ellipsis button and then select Delete.
    It can take up to 30 minutes to remove a traffic group. Removing the traffic group removes the T0 router that was created to support it. HCX, if in use, creates its own association map, which you can view but not modify. To remove an association map created by HCX, you have to uninstall HCX. See Uninstalling VMware HCX in the VMware HCX User Guide.

Example: Route Table Changes After Adding a Traffic Group

This simplified example shows the effect of creating traffic group and associating it with a prefix list of just two host routes (/32).

Initial configuration
Assume these values for route table entries in the default traffic group and the Compute Gateway (CGW) before adding the first traffic group (which creates an additional T0 router).
Table 1. Default Routes
Subnet Next Hop
0.0.0.0/0 Internet Gateway
192.168.150.51/24 CGW
192.168.151.0/24 CGW
VTGW, DXGW subnets VTGW, DXGW connections
Management CIDR MGW
Table 2. CGW Routes With the Default Traffic Group
Subnet Next Hop
0.0.0.0/0 Default T0
192.168.150.0/24 Default T0
192.168.151.0/24 Default T0
Multi-Edge configuration
After the first traffic group is created, new routes are added on the default T0. Assuming that the prefix list associated with the traffic group has these entries:
192.168.150.100/32
192.168.151.51/32
then the route tables for the default T0, new T0, and CGW end up like this.
Table 3. Default T0 Routes After Adding a Traffic Group
Subnet Next Hop
0.0.0.0/0 Internet Gateway
192.168.150.0/24 CGW
192.168.150.100/32 New T0
192.168.151.0/24 CGW
192.168.151.51/32 New T0
VTGW, DXGW subnets VTGW, DXGW connections
Management CIDR MGW
The new routes (192.168.150.100/32 and 192.168.151.51/32 in the example tables) use the new T0 as their next-hop, and the new T0 uses longest-prefix matching to route that traffic to the CGW.
Table 4. Routes on the New Traffic Group
Subnet Next Hop
0.0.0.0/0 Default T0
192.168.150.100/32 CGW
192.168.151.51/32 CGW
VTGW, DXGW subnets VTGW, DXGW connections
Management CIDR MGW
The CGW route table is updated to create the traffic group by specifying the new T0 router as the next hop for the new routes.
Table 5. CGW Routes With an Additional Traffic Group
Subnet Next Hop
0.0.0.0/0 Default T0
192.168.150.0/24 Default T0
192.168.150.100/32 New T0
192.168.151.0/24 Default T0
192.168.151.51/32 New T0