Before you start deploying Cloud Foundation, you must configure your physical network.

Prior to deploying Cloud Foundation, configure your physical network to enable the following functionality.
Table 1. Physical Network Requirements
Requirement Feature
IGMP snooping querier Required for the following traffic types:
  • VXLAN
Jumbo frames Required for the following traffic types:
  • vSAN
  • vSphere vMotion
  • VXLAN
  • NFS
A minimum MTU value of 1600 is required, however it is recommended that you set the MTU to 9000.
BGP adjacency and BGP autonomous system (AS) numbers Dynamic routing in the SDDC

Prepare your top of rack (ToR) switches by configuring Border Gateway Protocol (BGP) on the switches, defining the Autonomous System (AS) number and Router ID, and creating interfaces to connect with Edge Services Gateways (ESGs).

You will need the BGP Autonomous System (AS) number and the Router ID for each ToR switch when you deploy Cloud Foundation 3.9.1.

Physical Network Design

Design of the physical SDDC network includes defining the network topology for connecting the physical switches and the ESXi hosts, determining switch port settings for VLANs and link aggregation, and designing routing. VMware Cloud Foundation can use most enterprise grade physical network architectures.

Switch Types and Network Connectivity

Follow network vendor best practices for physical connectivity as well as configuration of VLANs and switch-port port settings.

Top of Rack Physical Switches

When configuring top of rack (ToR) switches, consider the following best practices.
  • Configure redundant physical switches to enhance availability.
  • Configure switch ports that connect to ESXi hosts manually as trunk ports. Virtual switches are passive devices and do not support trunking protocols, such as Dynamic Trunking Protocol (DTP).
  • Modify the Spanning Tree Protocol (STP) on any port that is connected to an ESXi NIC to reduce the time it takes to transition ports over to the forwarding state, for example, using the Trunk PortFast feature on a Cisco physical switch.
  • Provide DHCP or DHCP Helper capabilities on all VLANs that are used by the management and VXLAN VMkernel ports. This setup simplifies the configuration by using DHCP to assign IP address based on the IP subnet in use.
  • Configure jumbo frames on all switch ports, inter-switch link (ISL) and switched virtual interfaces (SVIs).

Top of Rack Connectivity and Network Settings

Each ESXi host is connected redundantly to the SDDC network fabric ToR switches by using a minimum of two 10-GbE ports (two 25-GbE or faster ports are recommended). Configure the ToR switches to provide all necessary VLANs via an 802.1Q trunk. These redundant connections use features of vSphere Distributed Switches and NSX Data Center for vSphere to guarantee that no physical interface is overrun and redundant paths are used as long as they are available.