When performing network configuration, you have to consider the overall traffic and decide how to isolate vSAN traffic.

vSAN Network Considerations

  • Consider how much replication and communication traffic is running between ESXi hosts. With vSAN, the amount of traffic depends on the number of VMs that are running in the cluster, and on how write-intensive the I/O is for the applications running in the VMs. 

The vSAN VMkernel port group is created as part of cluster creation. Configure this port group on all ESXi hosts in a cluster, even for ESXi hosts that are not contributing storage resources to the cluster. 

Figure 1. vSAN Conceptual Network

Network Bandwidth Requirements

For solutions, use a minimum of a 10-GbE connection, with 25-GbE recommended, for use with vSAN to ensure the best and most predictable performance (IOPS) for the environment. Without it, a significant decrease in array performance appears.

A minimum of 10-Gb Ethernet also provides support for future use of vSAN all-flash configurations.

Table 1. Network Speed Selection

Design Quality

1 GbE

10 GbE or Greater

Comments

Availability

o

o

Neither design option impacts availability.

Manageability

o

o

Neither design option impacts manageability.

Performance

Faster network speeds increase vSAN performance (especially in I/O intensive situations).

Recoverability

Faster network speeds increase the performance of rebuilds and synchronizations in the environment. As a result, VMs are properly protected from failures.

Security

o

o

Neither design option impacts security.

Legend: ↑ = positive impact on quality; ↓ = negative impact on quality; o = no impact on quality.

Table 2. Design Decisions on Network Bandwidth for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

CSDDC-VI-Storage-SDS-001

Use a minimum of 10 GbE (25 GbE recommended) for vSAN traffic.

Performance with 10 GbE is sufficient, while with 25 GbE is optimal. If the bandwidth is less than 10 GbE, array performance significantly decreases.

The physical network must support 10 Gb networking between every ESXi host in the vSAN clusters.

VMware vSAN Virtual Switch Type

vSAN supports the use of vSphere Standard Switch or vSphere Distributed Switch. The benefit of using vSphere Distributed Switch is that it supports Network I/O Control which supports prioritization of bandwidth if contention occurs.

This design uses a vSphere Distributed Switch for the vSAN port group to ensure that priority can be assigned using Network I/O Control to separate and guarantee the bandwidth for vSAN traffic.

Virtual Switch Design Background

Virtual switch type affects performance and security of the environment.

Table 3. Virtual Switch Types

Design Quality

vSphere Standard Switch

vSphere Distributed Switch

Comments

Availability

o

o

Neither design option impacts availability.

Manageability

The vSphere Distributed Switch is centrally managed across all ESXi hosts, unlike the standard switch which is managed on each ESXi host individually.

Performance

The vSphere Distributed Switch has added controls, such as Network I/O Control, which you can use to guarantee performance for vSAN traffic.

Recoverability

The vSphere Distributed Switch configuration can be backed up and restored, the standard switch does not have this functionality.

Security

The vSphere Distributed Switch has added built-in security controls to help protect traffic.

Legend: ↑ = positive impact on quality; ↓ = negative impact on quality; o = no impact on quality.

Table 4. Design Decisions on Virtual Switch Configuration for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

CSDDC-VI-Storage-SDS-002

Use the existing vSphere Distributed Switch instance.

Provides guaranteed performance for vSAN traffic, if there is network contention, by using existing networking components. 

All traffic paths are shared over common uplinks.

Jumbo Frames

vSAN supports jumbo frames for vSAN traffic. 

A vSAN design should use jumbo frames only if the physical environment is already configured to support them, they are part of the existing design, or if the underlying configuration does not create a significant amount of added complexity to the design.

Table 5. Design Decisions on Jumbo Frames for vSAN

Decision ID

Design Decision

Design Justification

Design Implication

CSDDC-VI-Storage-SDS-003

Configure jumbo frames on the VLAN dedicated to vSAN traffic.

Jumbo frames are already used to improve performance of vSphere vMotion and NFS storage traffic.

Every device in the network must support jumbo frames.

VLANs

Isolate vSAN traffic on its own VLAN. When a design uses multiple vSAN clusters, each cluster should use a dedicated VLAN or segment for its traffic. This approach prevents interference between clusters and helps with troubleshooting cluster configuration.

Table 6.  Design Decisions on vSAN VLAN

Decision ID

Design Decision

Design Justification

Design Implication

CSDDC-VI-Storage-SDS-004

Use a dedicated VLAN for vSAN traffic for each vSAN enabled cluster.

VLANs provide traffic isolation.

VLANs span only a single cluster.

Enough VLANs are available in each cluster and are to be used for traffic separation.