You can separate data traffic from witness traffic in two-node vSAN clusters and stretched clusters.

vSAN data traffic requires a low-latency, high-bandwidth link. Witness traffic can use a high-latency, low-bandwidth and routable link. To separate data traffic from witness traffic, you can configure a dedicated VMkernel network adapter for vSAN witness traffic.

You can add support for a direct network cross-connection to carry vSAN data traffic in a vSAN stretched cluster. You can configure a separate network connection for witness traffic. On each data host in the cluster, configure the management VMkernel network adapter to also carry witness traffic. Do not configure the witness traffic type on the witness host.

Note: Network Address Translation (NAT) is not supported between vSAN data hosts and the witness host.

Prerequisites

  • Verify that the data site to witness traffic connection has a minimum bandwidth of 2 Mbps for every 1,000 vSAN components.
  • Verify the latency requirements:
    • Two-node vSAN clusters must have less than 500 ms RTT.
    • Stretched clusters with less than 11 hosts per site must have less than 200 ms RTT.
    • Stretched clusters with 11 or more hosts per site must have less than 100 ms RTT.
  • Verify that the vSAN data connection meets the following requirements.
    • For hosts directly connected in a two-node vSAN cluster, use a 10 Gbps direct connection between hosts. Hybrid clusters also can use a 1 Gbps crossover connection between hosts.
    • For hosts connected to a switched infrastructure, use a 10 Gbps shared connection (required for all-flash clusters), or a 1 Gbps dedicated connection.
  • Verify that data traffic and witness traffic use the same IP version.

Procedure

  1. Open an SSH connection to the ESXi host.
  2. Use the esxcli network ip interface list command to determine which VMkernel network adapter is used for management traffic.
    For example:
    esxcli network  ip interface list
    vmk0
       Name: vmk0
       MAC Address: e4:11:5b:11:8c:16
       Enabled: true
       Portset: vSwitch0
       Portgroup: Management Network
       Netstack Instance: defaultTcpipStack
       VDS Name: N/A
       VDS UUID: N/A
       VDS Port: N/A
       VDS Connection: -1
       Opaque Network ID: N/A
       Opaque Network Type: N/A
       External ID: N/A
       MTU: 1500
       TSO MSS: 65535
       Port ID: 33554437
    
    vmk1
       Name: vmk1
       MAC Address: 00:50:56:6a:3a:74
       Enabled: true
       Portset: vSwitch1
       Portgroup: vsandata
       Netstack Instance: defaultTcpipStack
      VDS Name: N/A
       VDS UUID: N/A
       VDS Port: N/A
       VDS Connection: -1
       Opaque Network ID: N/A
       Opaque Network Type: N/A
       External ID: N/A
       MTU: 9000
       TSO MSS: 65535
       Port ID: 50331660
    
    Note: Multicast information is included for backward compatibility. vSAN 6.6 and later releases do not require multicast.
  3. Use the esxcli vsan network ip add command to configure the management VMkernel network adapter to support witness traffic.
    esxcli vsan network ip add -i vmkx -T witness 
  4. Use the esxcli vsan network list command to verify the new network configuration.
    For example:
    esxcli vsan network list
    Interface
       VmkNic Name: vmk0
       IP Protocol: IP
       Interface UUID: 8cf3ec57-c9ea-148b-56e1-a0369f56dcc0
       Agent Group Multicast Address: 224.2.3.4
       Agent Group IPv6 Multicast Address: ff19::2:3:4
       Agent Group Multicast Port: 23451
       Master Group Multicast Address: 224.1.2.3
       Master Group IPv6 Multicast Address: ff19::1:2:3
       Master Group Multicast Port: 12345
       Host Unicast Channel Bound Port: 12321
       Multicast TTL: 5
       Traffic Type: witness
    
    Interface
       VmkNic Name: vmk1
       IP Protocol: IP
       Interface UUID: 6df3ec57-4fb6-5722-da3d-a0369f56dcc0
       Agent Group Multicast Address: 224.2.3.4
       Agent Group IPv6 Multicast Address: ff19::2:3:4
       Agent Group Multicast Port: 23451
       Master Group Multicast Address: 224.1.2.3
       Master Group IPv6 Multicast Address: ff19::1:2:3
       Master Group Multicast Port: 12345
       Host Unicast Channel Bound Port: 12321
       Multicast TTL: 5
       Traffic Type: vsan
    

Results

In the vSphere Client, the management VMkernel network interface is not selected for vSAN traffic. Do not re-enable the interface in the vSphere Client.