VMware Cloud on AWS uses NSX-T to create and manage internal SDDC networks and provide endpoints for VPN connections from your on-premises network infrastructure.

SDDC Network Topology

When you create an SDDC, it includes a Management Network. Single-host trial SDDCs also include a small Compute Network. You specify the Management Network CIDR block when you create the SDDC. It cannot be changed after the SDDC has been created. See Deploy an SDDC from the VMC Console for details. The Management Network has two subnets:
Appliance Subnet
This subnet is used by the vCenter, NSX, and HCX appliances in the SDDC. When you add appliance-based services such as SRM to the SDDC, they also connect to this subnet.
Infrastructure Subnet
This subnet is used by the ESXi hosts in the SDDC.

The Compute Network includes an arbitrary number of logical segments for your workload VMs. See VMware Configuration Maximums for current limits on logical segments. In a Single Host SDDC starter configuration, we create a compute network with a single routed segment. In SDDC configurations that have more hosts, you'll have to create compute network segments to meet your needs. See VMware Configuration Maximums for applicable limits.

An SDDC network has two notional tiers:
  • Tier 0 handles north-south traffic (traffic leaving or entering the SDDC, or between the Management and Compute gateways)
  • Tier 1 handles east-west traffic (traffic between routed network segments within the SDDC).
Figure 1. SDDC Network Topology
NSX Edge Appliance

The default NSX Edge Appliance is implemented as a pair of VMs that run in active/standby mode. This appliance provides the platform on which the default Tier 0 and Tier 1 routers run, along with IPsec VPN connections and their BGP routing machinery. All north-south traffic goes through the Tier 0 router. To avoid sending east-west traffic through the Edge Appliance, a component of each Tier 1 router runs on every ESXi host that handles routing for destinations within the SDDC.

If you need additional bandwidth for the subset of this traffic that doesn't travel over a VPN or take any other route to or from the Internet, you can reconfigure your SDDC to be Multi-Edge by creating traffic groups, each of which creates an additional NSX Edge Appliance. See Configure a Multi-Edge SDDC With Traffic Groups for details.

Management Gateway (MGW)
The MGW is a Tier 1 router that handles routing and firewalling for the vCenter Server and other management appliances running in the SDDC. The Internet-facing IP address of the MGW is automatically assigned from the pool of AWS public IP addresses when the SDDC is created. See Deploy an SDDC from the VMC Console for more about specifying this address range.
Compute Gateway (CGW)
The CGW is a Tier 1 router that handles routing and firewalling for workload VMs connected to routed compute network segments.

Routing Between Your SDDC and the Connected VPC


Any VPC Subnets on which AWS services or instances communicate with the SDDC must be associated with the main route table of the connected VPC. Use of a custom route table or replacement of the main route table is not supported.

When you create an SDDC, we pre-allocate 17 AWS Elastic Network Interfaces (ENIs) in the selected VPC owned by the AWS account you specify at SDDC creation. We assign each of these ENIs an IP address from the subnet you specify at SDDC creation, then attach each of the hosts in the SDDC cluster Cluster-1 to one of these ENIs. An additional IP address is assigned to the ENI where the active NSX Edge Appliance is running.

This configuration, known as the Connected VPC, supports network traffic between VMs in the SDDC and AWS instances and native AWS service endpoints in the Connected VPC. The main route table of the connected VPC is aware of the VPC’s primary subnet as well as all SDDC (NSX-T network segment) subnets. When you create or delete routed network segments on the SDDC, the main route table is automatically updated. When the NSX Edge Appliance in your SDDC is moved to another host, either to recover from a failure or during SDDC maintenance, the IP address allocated to the Edge Appliance is moved to the new ENI (on the new host), and the main route table is updated to reflect the change. If you have replaced the main route table or are using a custom route table, that update fails and network traffic can no longer be routed between SDDC networks and the Connected VPC. See View Connected VPC Information for more about how to use the VMC Console to see the details of your connected VPC.

For an in-depth discussion of SDDC network architecture and the AWS network objects that support it, read the VMware Cloud Tech Zone article VMware Cloud on AWS: SDDC Network Architecture .

Reserved Network Addresses

Certain IPv4 address ranges are unavailable for use in SDDC compute networks. Several are used internally by SDDC network components. Most are reserved by convention on other networks as well.
Table 1. Reserved Address Ranges in SDDC Networks
These ranges are reserved within the SDDC management subnet, but can be used in your on-premises networks or SDDC compute network segments. Reserved for carrier-grade NAT per RFC 6598. Avoid using addresses in this range in SDDC networks and others. They are not likely to be reachable within the SDDC or from outside it. See VMware Knowledge Base article 76022 for a detailed breakdown of how SDDC networks use this address range.
Per RFC 3927, all of is a link-local range that cannot be routed beyond a single subnet. However, with the exception of these CIDR blocks, you can use addresses for your virtual tunnel interfaces. See Create a Route-Based VPN. This the default compute segment CIDR for a single-host starter SDDC and is not reserved in other configurations.
SDDC networks also observe the conventions for special Use IPv4 address ranges enumerated in RFC 3330.

Multicast Support in SDDC Networks

In SDDC networks, layer 2 multicast traffic is treated as broadcast traffic on the network segment where the traffic originates. It is not routed beyond that segment. Layer 2 multicast traffic optimization features such as IGMP snooping are not supported. Layer 3 multicast (such as Protocol Independent Multicast) is not supported in VMware Cloud on AWS.

Connecting Your On-Premises SDDC to Your Cloud SDDC

To connect your on-premises data center to your VMware Cloud on AWS SDDC, you can create a VPN that uses the public Internet, a VPN that uses AWS Direct Connect, or just use AWS Direct Connect alone. You can also take advantage of SDDC groups to use VMware Transit Connect™ and an AWS Direct Connect Gateway to provide centralized connectivity between a group of VMware Cloud on AWS SDDCs and an on-premises SDDC. See Creating and Managing SDDC Deployment Groups in the VMware Cloud on AWS Operations Guide.
Figure 2. SDDC Connections to your On-Premises Data Center
Layer 3 (L3) VPN
A layer 3 VPN provides a secure connection between your on-premises data center to your VMware Cloud on AWS SDDC over the public Internet or AWS Direct Connect. These IPsec VPNs can be either route-based or policy-based. You can create up to sixteen VPNs of each type, using any on-premises router that supports the settings listed in the IPsec VPN Settings Reference as the on-premises endpoint.
Layer 2 (L2) VPN
A layer 2 VPN provides an extended, or stretched, network with a single IP address space that spans your on-premises data center and your SDDC and enables hot or cold migration of on-premises workloads to the SDDC. You can create only a single L2VPN tunnel in any SDDC. The on-premises end of the tunnel requires NSX. If you are not already using NSX in your on-premises data center, you can download a standalone NSX Edge appliance to provide the required functionality. An L2 VPN can connect your on-premises data center to the SDDC over the public Internet or over AWS Direct Connect.
AWS Direct Connect (DX)
AWS Direct Connect is a service provided by AWS that allows you to create a high-speed, low latency connection between your on-premises data center and AWS services. When you configure AWS Direct Connect, VPNs can use it instead of routing traffic over the public Internet. Because Direct Connect implements Border Gateway Protocol (BGP) routing, use of an L3VPN for the management network is optional when you configure Direct Connect. Traffic over Direct Connect is not encrypted. If you want to encrypt that traffic, you can configure an IPsec VPN that uses private IP addresses and Direct Connect.
VMware HCX
VMware HCX, a multi-cloud app mobility solution, is provided free to all SDDCs and facilitates migration of workload VMs to and from your on-premises data center to your SDDC. For more information about installing, configuring, and using HCX, see the Hybrid Migration with HCX Checklist.