Your DX connection requires a private virtual interface to enable vMotion, ESXi Management, Management Appliance, and workload traffic to use it.

Create one private virtual interface (VIF) for each Direct Connect link you want to make to your SDDC. For example, if you want to create two Direct Connect links for redundancy, create two private VIFs in the AWS account linked to your SDDC. See VMware Configuration Maximums for limits on the number of segments supported by each private VIF.

When you create a private VIF in the AWS Account ID shown on the Direct Connect page of the Networking & Security tab, it can be attached to any of your Organization’s SDDCs in the region where you created the VIF. After you attach it to an SDDC, the VIF cannot be detached or reassigned to another SDDC. Instead, it must be deleted and a new VIF created. Deleting an SDDC deletes any attached VIFs.

Important:

When you connect a DX private virtual interface or a VTGW to an SDDC, all outbound traffic from ESXi hosts to destinations outside the SDDC network is routed over that interface, regardless of other routing configurations in the SDDC. This includes vMotion and vSphere replication traffic. You must ensure that inbound traffic to ESXi hosts is also routed over the DX interface so that the inbound and outbound traffic paths are symmetrical. See Creating and Managing SDDC Deployment Groups with VMware Transit Connect in the VMware Cloud on AWS Operations Guide for more about VMware Transit Connect and the VMware Managed Transit Gateway (VTGW).

Although routes learned from a route-based VPN are advertised over BGP to other route-based VPNs, an SDDC advertises only its own networks over DX, not any learned from VPNs. See AWS Direct Connect quotas in the AWS Direct Connect User Guide for detailed information about limits imposed by AWS on Direct Connect, including limits on routes advertised and learned over BGP.

Prerequisites

Procedure

  1. Log in to VMware Cloud Services at https://vmc.vmware.com.
  2. Click Inventory > SDDCs, then pick an SDDC card and click VIEW DETAILS.
  3. Click OPEN NSX MANAGER and log in with the NSX Manager Admin User Account shown on the SDDC Settings page.
    You can also use the VMC Console Networking & Security tab for this workflow. See SDDC Network Administration with NSX Manager.
  4. Log in to the AWS Console and complete the Creating a Hosted Private Virtual Interface procedure under Create a Hosted Virtual Interface.
    If you're using a hosted VIF, work with your AWS Direct Connect Partner to create the VIF in the account shown in the AWS Account ID field of the Direct Connect page, then skip to Step 5 of this procedure. If you are using a dedicated or hosted connection, take these steps first.
    1. For Virtual interface type, choose Private and make up a Virtual interface name.
    2. For the Virtual interface Owner field, select Another AWS account and use the AWS Account ID from the NSX Direct Connect page.
    3. For VLAN, use the value provided by your AWS Direct Connect Partner.
    4. For BGP ASN, use the ASN of the on-premises router where this connection terminates.
      This value must not be the same as the BGP Local ASN shown on the NSX Direct Connect page.
    5. Expand Additional Settings and make the following choices:
      Address family Select IPV4
      Your router peer ip Specify the IP address of the on-premises end of this connection (your router), or leave blank to have AWS automatically assign an address that you'll need to configure in your router.
      Amazon router peer ip Specify the IP address of the AWS end of this connection, or leave blank to have AWS automatically assign an address that you'll need to configure in your router.
      BGP authentication key Specify a value or leave blank to have AWS generate a key, which you'll need to configure in your router.
      Jumbo MTU (MTU size 9001) The default MTU for all SDDC networks is 1500 bytes. To enable DX traffic to this private VIF to use a larger MTU, select Enable under Jumbo MTU (MTU size 9001). After the VIF has been created, you'll also need to open the NSX Global Configuration page and set a higher MTU value under Intranet Uplink, as described in Specify the Direct Connect MTU. Enabling this in the connection properties, even if you don't intend to use it right away, makes it easier to take advantage of jumbo frames in SDDC networks when you need them.
    When the interface has been created, the AWS console reports that it is ready for acceptance.
  5. Open NSX Manager or the VMC Console Networking & Security tab. Click Direct Connect and accept the virtual interface by clicking ATTACH.
    Before it has been accepted, a new VIF is visible in all SDDCs in your organization. After you accept the VIF, it is no longer visible in any other SDDC.
    It can take up to 10 minutes for the BGP session to become active. When the connection is ready, the State shows as Attached and the BGP Status as Up.
  6. (Optional) Configure a route-based VPN as the backup to Direct Connect.
    In the default configuration, traffic on any route advertised over BGP by both DX and a route-based VPN uses the VPN by default. To have a route advertised by both DX and VPN use DX by default and failover to the VPN when DX is unavailable click Direct Connect and set the Use VPN as backup to Direct Connect switch to Enabled.
    Note: This configuration requires a route-based VPN. You cannot use a policy-based VPN as a backup to Direct Connect. In an SDDC that is a member of an SDDC group, traffic over a route that is advertised by both the DX private VIF and the group's VMware Managed Transit Gateway ( VTGW) will be routed over the VTGW.
    The system requires a minute or so to update your routing preference. When the operation completes, routes advertised by both DX and VPN default to the DX connection, using the VPN only when DX is unavailable. Equivalent routes advertised by both DX and VPN prioritize the VPN connection.

Results

A list of Advertised BGP Routes and Learned BGP Routes is displayed as the routes are learned and advertised. Click the refresh icon to refresh these lists. All routed subnets in the SDDC are advertised as BGP routes, along with this subset of management network subnets:
  • Subnet 1 includes routes used by ESXi host vmks and router interfaces.
  • Subnet 2 includes routes used for Multi-AZ support and AWS integration.
  • Subnet 3 includes management VMs.
Disconnected and extended networks are not advertised.

The actual CIDR blocks advertised depend on your management subnet CIDR block. The following table shows the CIDR blocks for these routes in an SDDC that uses the default management network CIDR of 10.2.0.0 in block sizes /16, /20, and /22.

Table 1. Advertised Routes for 10.2.0.0 Default MGW CIDR
MGW CIDR Subnet 1 Subnet 2 Subnet 3
10.2.0.0/23 10.2.0.0/24 10.2.1.0/26 10.2.1.128/25
10.2.0.0/20 10.2.0.0/21 10.2.8.0/23 10.2.12.0/22
10.2.0.0/16 10.2.0.0/17 10.2.128.0/19 10.2.192.0/18

What to do next

Ensure the vMotion interfaces are configured to use Direct Connect. See Configure vMotion Interfaces for Use with Direct Connect.