You can implement the switch fabric at the physical layer by providing Layer 2 or Layer 3 transport services. For a scalable and vendor-neutral data solution, use a Layer 3 transport. Both layer 2 and layer 3 transport have benefits and drawbacks. When designing the network architecture, consider the following for Layer 2 and Layer 3 transport.

Layer 2 Transport Considerations

The following considerations apply to a design that uses Layer 2 transport:

  • Top-of-Rack (ToR) switches and upstream Layer 3 devices such as core switches and routers form a switched fabric.

  • The upstream Layer 3 devices terminate each VLAN and provide the default gateway functionality.

  • Uplinks from the ToR switch to the upstream Layer 3 devices are 802.1Q trunks carrying all required VLANs.

Figure 1. Layer 2 Transport
Layer 2 Transport
Table 1. Benefits and Drawbacks of a Layer 2 Transport

Characteristic

Description

Benefits

  • Additional design flexibility.

  • You can span VLANs across racks.

Drawbacks

  • The size of this deployment is limited because the fabric elements share a limited number of VLANs.

  • You might have to rely on a specialized switching fabric product from a single vendor.
Layer 3 Transport Considerations

The following considerations apply to a design that uses Layer 3 transport:

  • Layer 2 connectivity is limited to the ToR switches.

  • The ToR switch terminates each VLAN and provides the default gateway functionality. That is, it has a Switch Virtual Interface (SVI) for each VLAN.

  • Uplinks from the ToR switch to the upstream layer are routed point-to-point links. You cannot use VLAN trunking on the uplinks.

  • A dynamic routing protocol such as external Border Gateway Protocol (eBGP) connects the ToR switches and upstream switches. Each ToR switch advertises the prefixes, typically one per VLAN or subnet. In turn, the ToR switch calculates equal-cost paths to the prefixes received from the upstream layer it peers with.

Figure 2. Layer 3 Transport
Layer 3 Transport
Table 2. Benefits and Drawbacks of a Layer 3 Transport

Characteristic

Description

Benefits

  • You can select from many Layer 3 capable switch products for the physical switching fabric.

  • You can mix switches from different vendors because of the general interoperability between the implementation of routing protocols.

  • This approach is cost-effective because it uses only the basic functionality of the physical switches.

Drawbacks

  • VLANs are restricted to a single rack. The restriction can affect IP-based storage networks, such as iSCSI and NFS.

Infrastructure Networks

Each ESXi host has multiple VMkernel port groups that are configured as infrastructure networks. The infrastructure networks include:

  • ESXi management network to handle the ESXi host management traffic.

  • vMotion network to handle the VMware vSphere vMotion® traffic.

  • vSAN network to handle the vSAN shared-storage traffic.

  • NFS network (optional) dedicated to offline storage, such as NFS, and used for backups and restores for the workloads that need it.

Management Networks

Management networks interconnect the VMs of the VMware Telco Cloud platform. The management networks include:

  • A dedicated VLAN for management component communication. DHCP is required for installing Kubernetes management nodes. Configure DHCP service and IP pool for management VLAN network and enable internet access.

  • Operations and Management networks (OAM) for

    fault, configuration, accounting, performance, and security