VMware Cloud on AWS uses NSX-T to create and manage internal SDDC networks and provide endpoints for VPN connections from your on-premises network infrastructure.

Connecting to your SDDC

To connect your on-premises data center to your VMware Cloud on AWS SDDC, you can create a VPN that uses the public Internet, a VPN that uses AWS Direct Connect, or just use AWS Direct Connect alone.
Figure 1. SDDC Connections to your On-Premises Data Center
Layer 3 (L3) VPN
A layer 3 VPN provides a management network that connects your on-premises data center to your SDDC. These IPsec VPNs can be either route-based or policy-based. You can create up to sixteen VPNs of each type, using any on-premises router that supports the settings listed in the IPsec VPN Settings Reference. An L3 VPN can connect your on-premises data center to the SDDC over the public Internet or over AWS Direct Connect.
Layer 2 (L2) VPN
A layer 2 VPN provides an extended, or stretched, network with a single IP address space that spans your on-premises data center and your SDDC and enables hot or cold migration of on-premises workloads to the SDDC. You can create only a single L2VPN tunnel in any SDDC. The on-premises end of the tunnel requires NSX. If you are not already using NSX in your on-premises data center, you can download a standalone NSX Edge appliance to provide the required functionality. An L2 VPN can connect your on-premises data center to the SDDC over the public Internet or over AWS Direct Connect.
AWS Direct Connect (DX)
AWS Direct Connect is a service provided by AWS that allows you to create a high-speed, low latency connection between your on-premises data center and AWS services. When you configure AWS Direct Connect, VPNs can use it instead of routing traffic over the public Internet. Because Direct Connect implements Border Gateway Protocol (BGP) routing, use of an L3VPN for the management network is optional when you configure Direct Connect. Traffic over Direct Connect is not encrypted. If you want to encrypt that traffic, configure your L3 VPN to use Direct Connect.
VMware HCX
VMware HCX, a multi-cloud app mobility solution, is provided free to all SDDCs and facilitates migration of workload VMs to and from your on-premises data center to your SDDC. For more information about installing, configuring, and using HCX, see the Hybrid Migration with HCX Checklist.

SDDC Network Topology

When you create an SDDC, it includes a Management Network and a Compute Network. The Management Network CIDR block must be specified when you created the SDDC and cannot be changed. The Management Network has two subnets:
Appliance Subnet
A subnet of the CIDR range you specified for the Management Subnet when you created the SDDC. This subnet is used by the vCenter, NSX, and HCX appliances in the SDDC. When you add appliance-based services such as SRM to the SDDC, they also connect to this subnet.
Infrastructure Subnet
A subnet of the CIDR range you specified for the Management Subnet when you created the SDDC. This subnet is used by the ESXi hosts in the SDDC.

The compute network can have up to 16 segments for your workload VMs. In a Single Host SDDC starter configuration, we create a compute network with a single routed segment. In SDDC configurations that have more hosts, you'll have to create compute network segments to meet your needs.

A Tier 0 NSX Edge appliance sits between your on-premises networks and your SDDC networks, and routes traffic to either the management network or the compute network as appropriate.

Figure 2. SDDC Network Topology
Tier 0 Edge Appliance
All traffic between your on-premises networks and the SDDC passes through this appliance. Compute Gateway firewall rules, which control access to workload VMs, are applied on its uplink interfaces.
Management Gateway (MGW)
The MGW is an NSX Edge Security gateway that provides north-south network connectivity for the vCenter Server and other management appliances running in the SDDC. The Internet-facing IP address (Public IP #1) is automatically assigned from the pool of AWS public IP addresses when the SDDC is created. Pick an address range (CIDR block) for the management subnet that can support the number of ESXi hosts in your SDDC. If you don't specify a range when you create the SDDC, the system uses a default of 10.2.0.0/16.
Compute Gateway (CGW)
The CGW provides north-south network connectivity for virtual machines running in the SDDC. In a single-node SDDC, VMware Cloud on AWS creates a default logical network segment (CIDR block 192.168.1.0/24) to provide networking for these VMs. You can create additional logical networks on the Networking & Security tab.

Routing Between Your SDDC and the Connected VPC

Important:

All SDDC subnets and any VPC subnets on which AWS services or instances communicate with the SDDC must be associated with the main route table of the connected VPC. Use of a custom route table or replacement of the main route table is not supported.

When you create an SDDC, we connect the ENI of the VPC owned by the AWS account you specify to the NSX Edge Appliance in the SDDC. That VPC becomes the Connected VPC, and the connection supports network traffic between SDDC VMs and AWS instances and native services in the Connected VPC. The main route table of the connected VPC includes all the subnets in the VPC as well as all SDDC (NSX-T network segment) subnets. When you create or delete routed network segments on the workload network, the main route table is automatically updated. When the NSX Edge Appliance in your SDDC is moved to another host, either to recover from a failure or during SDDC maintenance, the main route table is updated to reflect the ENI used by the new NSX Edge host. If you have replaced the main route table or are using a custom route table, that update fails and network traffic can no longer be routed between SDDC networks and the Connected VPC.

For more information, see View Connected VPC Information.

Multicast Support in SDDC Networks

In SDDC networks, layer 2 multicast traffic is treated as broadcast traffic on the network segment where the traffic originates. It is not routed beyond that segment. Layer 2 multicast traffic optimization features such as IGMP snooping are not supported. Layer 3 multicast (such as Protocol Independent Multicast) is not supported in VMware Cloud on AWS.