In the network design for vSAN in the management domain, you determine the network configuration for vSAN traffic.

Consider the overall traffic bandwidth and decide how to isolate storage traffic.

  • Consider how much vSAN data traffic is running between ESXi hosts.

  • The amount of storage traffic depends on the number of VMs that are running in the cluster, and on how write- intensive the I/O process is for the applications running in the VMs.

For information on the physical network setup for vSAN traffic, and other system traffic, see Physical Network Infrastructure Design for NSX-T Data Center for the Management Domain.

For information on the virtual network setup for vSAN traffic, and other system traffic, see Distributed Port Group and VMkernel Adapter Design for the Management Domain.

vSAN Network Design

The vSAN network design includes these components.

Table 1. Components of vSAN Network Design

Design Component

Description

Physical NIC speed

For best and most predictable performance (IOPS) for the environment, this design uses a minimum of a 10-GbE connection, with 25-GbE recommended, for use with vSAN all-flash configurations.

VMkernel network adapters for vSAN

The vSAN VMkernel network adapter on each ESXi host is created when you enable vSAN on the cluster. Connect the vSAN VMkernel network adapters on all ESXi hosts in a cluster to a dedicated distributed port group including also ESXi hosts that are not contributing storage resources to the cluster.

VLAN

All storage traffic should be isolated on its own VLAN. When a design uses multiple vSAN clusters, each cluster should use a dedicated VLAN or segment for its traffic. This approach increases security, prevents interference between clusters and helps with troubleshooting cluster configuration.

Jumbo frames

vSAN traffic can be handled by using jumbo frames. Use jumbo frames for vSAN traffic only if the physical environment is already configured to support them, they are part of the existing design, or if the underlying configuration does not create a significant amount of added complexity to the design.
Table 2. Design Decisions on the Virtual Switch Configuration for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-NET-001

Use the existing vSphere Distributed Switch instances in the default management cluster.

Provides guaranteed performance for vSAN traffic in a connection-free network by using existing networking components. 

All traffic paths are shared over common uplinks.

VCF-MGMT-VSAN-NET-002

Configure jumbo frames on the VLAN for vSAN traffic.

  • Simplifies configuration because jumbo frames are also used to improve the performance of vSphere vMotion and NFS storage traffic.

  • Reduces the CPU overhead resulting high network usage.

Every device in the network must support jumbo frames.