When performing network configuration, you have to consider the overall traffic and decide how to isolate vSAN traffic.

vSAN Network Considerations

  • Consider how much replication and communication traffic is running between ESXi hosts. With vSAN, the amount of traffic depends on the number of VMs that are running in the cluster, and on how write-intensive the I/O is for the applications running in the VMs. 

  • Isolate vSAN traffic on its own Layer 2 network segment. You can do this using dedicated switches or ports, or by using a VLAN. 

The vSAN VMkernel port group is created as part of cluster creation. Configure this port group on all ESXi hosts in a cluster, even for ESXi hosts that are not contributing storage resources to the cluster. 

Figure 1. VMware vSAN Conceptual Network



Network Bandwidth Requirements

For solutions use a 10-Gb Ethernet connection for use with vSAN to ensure the best and most predictable performance (IOPS) for the environment. Without it, a significant decrease in array performance results.

Table 1. Network Speed Selection

Design Quality

1Gb

10Gb

Comments

Availability

o

o

Neither design option impacts availability.

Manageability

o

o

Neither design option impacts manageability.

Performance

Faster network speeds increase vSAN performance (especially in I/O intensive situations).

Recoverability

Faster network speeds increase the performance of rebuilds and synchronizations in the environment. This ensures that VMs are properly protected from failures.

Security

o

o

Neither design option impacts security.

 Legend: ↑ = positive impact on quality; ↓ = negative impact on quality; o = no impact on quality.

Note:

10 GbE Ethernet connection also provides support for future use of vSAN all-flash configurations.

Table 2. Design Decisions on Network Bandwidth for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

ROBO-VI-Storage-SDS-001

Use only 10 GbE for vSAN traffic.

Performance with 10 GbE is optimal. Without it, a significant decrease in array performance results.

The physical network must support 10 Gb networking between every ESXi host in the vSAN clusters.

VMware vSAN Virtual Switch Type

vSAN supports the use of vSphere Standard Switch or vSphere Distributed Switch. The benefit of using vSphere Distributed Switch is that it supports Network I/O Control which allows for prioritization of bandwidth in case of contention in an environment.

This design uses a vSphere Distributed Switch for the vSAN port group to ensure that priority can be assigned using Network I/O Control to separate and guarantee the bandwidth for vSAN traffic.

Virtual Switch Design Background

Virtual switch type affects performance and security of the environment.

Table 3. Virtual Switch Types

Design Quality

vSphere Standard Switch

vSphere Distributed Switch

Comments

Availability

o

o

Neither design option impacts availability.

Manageability

The vSphere Distributed Switch is centrally managed across all ESXi hosts, unlike the standard switch which is managed on each ESXi host individually.

Performance

The vSphere Distributed Switch has added controls, such as Network I/O Control, which you can use to guarantee performance for vSAN traffic.

Recoverability

The vSphere Distributed Switch configuration can be backed up and restored, the standard switch does not have this functionality.

Security

The vSphere Distributed Switch has added built-in security controls to help protect traffic.

Legend: ↑ = positive impact on quality; ↓ = negative impact on quality; o = no impact on quality.

Table 4. Design Decisions on Virtual Switch Configuration for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

ROBO-VI-Storage-SDS-002

Use the existing vSphere Distributed Switch instance.

Provides guaranteed performance for vSAN traffic, if there is network contention, by using existing networking components. 

All traffic paths are shared over common uplinks.

Jumbo Frames

VMware vSAN supports jumbo frames for vSAN traffic. 

A VMware vSAN design should use jumbo frames only if the physical environment is already configured to support them, they are part of the existing design, or if the underlying configuration does not create a significant amount of added complexity to the design.

Table 5. Design Decisions on Jumbo Frames for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

ROBO-VI-Storage-SDS-003

Configure jumbo frames on the VLAN dedicated to vSAN traffic.

Jumbo frames are already used to improve performance of vSphere vMotion and NFS storage traffic.

Every device in the network must support jumbo frames.

VLANs

VMware recommends isolating VMware vSAN traffic on its own VLAN. When a design uses multiple vSAN clusters, each cluster should use a dedicated VLAN or segment for its traffic. This approach prevents interference between clusters and helps with troubleshooting cluster configuration.

Table 6.  Design Decisions on vSAN VLAN

Decision ID

Design Decision

Design Justification

Design Implication

ROBO-VI-Storage-SDS-004

Use a dedicated VLAN for vSAN traffic for each vSAN enabled cluster.

VLANs provide traffic isolation.

VLANs span only a single cluster.  

Enough VLANs are available in each cluster and are to be used for traffic segregation.