Setup of the physical environment requires careful consideration. Follow best practices for physical switches, VLANs and subnets, and access port settings.

Top of Rack Physical Switches

When configuring Top of Rack (ToR) switches, consider the following best practices.

  • Configure redundant physical switches to enhance availability.

  • Configure switch ports that connect to ESXi hosts manually as trunk ports. Virtual switches are passive devices and do not send or receive trunking protocols, such as Dynamic Trunking Protocol (DTP).

  • Modify the Spanning Tree Protocol (STP) on any port that is connected to an ESXi NIC to reduce the time it takes to transition ports over to the forwarding state, for example using the Trunk PortFast feature found in a Cisco physical switch. 

  • Provide DHCP or DHCP Helper capabilities on all VLANs that are used by Management and VXLAN VMkernel ports. This setup simplifies the configuration by using DHCP to assign IP address based on the IP subnet in use.

  • Configure jumbo frames on all switch ports, inter-switch link (ISL) and switched virtual interfaces (SVI's).

Top of Rack Connectivity and Network Settings

Each ESXi host is connected redundantly to the SDDC network fabric ToR switches by means of two 10 GbE ports. Configure the ToR switches to provide all necessary VLANs via an 802.1Q trunk. 

VLANs and Subnets

Each ESXi host uses VLANs and corresponding subnets for internal-only traffic, as shown in Sample VLANs and Subnets within a Pod.

Follow these guidelines.

  • Use only /24 subnets to reduce confusion and mistakes when dealing with IPv4 subnetting.

  • Use the IP address .253 as the (floating) interface with .251 and .252 for Virtual Router Redundancy Protocol (VRPP) or Hot Standby Routing Protocol (HSRP).

  • Use the RFC1918 IPv4 address space for these subnets and allocate one octet by region and another octet by function. For example, the mapping 172.regionid.function.0/24 results in the following sample subnets.

Note:

The following VLANs and IP ranges are meant as samples. Your actual implementation depends on your environment.

Table 1. Sample VLANs and Subnets within a Pod

Pod

Function

Sample VLAN

Sample IP range

Consolidated

Management

1811 (Native)

172.18.11.0/24

Consolidated

vMotion

1812

172.18.12.0/24

Consolidated

vSAN

1813

172.18.13.0/24

Consolidated

VXLAN

1814

172.18.14.0/24

Consolidated

Storage

1815

172.18.15.0/24

Consolidated

Uplink 1

1816

172.18.16.0/24

Consolidated

Uplink 2

1817

172.18.17.0/24

Access Port Network Settings

Configure additional network settings on the access ports that connect the switch to the corresponding servers.

Spanning-Tree Protocol (STP)

Designate the access ports as trunk PortFast.

Trunking

Configure the VLANs as members of a 802.1Q trunk with the management VLAN acting as the native VLAN.

MTU

Set MTU for all VLANS and SVIs (Management, vMotion, VXLAN and Storage) to jumbo frames for consistency purposes.

DHCP helper

Configure the VIF of the Management and VXLAN subnet as a DHCP proxy.

Multicast

Configure IGMP snooping on the switches and include an IGMP querier on each VLAN.