You can separate data traffic from witness traffic in two-host vSAN clusters and stretched clusters.

vSAN data traffic requires a low-latency, high-bandwidth link. Witness traffic can use a high-latency, low-bandwidth and routable link. To separate data traffic from witness traffic, you can configure a dedicated VMkernel network adapter for vSAN witness traffic.

You can add support for a direct network cross-connection to carry vSAN data traffic in a vSAN stretched cluster. You can configure a separate network connection for witness traffic. On each data host in the cluster, configure the management VMkernel network adapter to also carry witness traffic. Do not configure the witness traffic type on the witness host.

Note: Network Address Translation (NAT) is not supported between vSAN data hosts and the witness host.

Prerequisites

  • Verify that the data site to witness traffic connection has a minimum bandwidth of 2 Mbps for every 1,000 vSAN components.
  • Verify the latency requirements:
    • Two-host vSAN clusters must have less than 500 ms RTT.
    • Stretched clusters with less than 11 hosts per site must have less than 200 ms RTT.
    • Stretched clusters with 11 or more hosts per site must have less than 100 ms RTT.
  • Verify that the vSAN data connection meets the following requirements.
    • For hosts directly connected in a two-host vSAN cluster, use a 10 Gbps direct connection between hosts. Hybrid clusters also can use a 1 Gbps crossover connection between hosts.
    • For hosts connected to a switched infrastructure, use a 10 Gbps shared connection (required for all-flash clusters), or a 1 Gbps dedicated connection.
  • Verify that data traffic and witness traffic use the same IP version.

Procedure

  1. Open an SSH connection to the ESXi host.
  2. Use the esxcli network ip interface list command to determine which VMkernel network adapter is used for management traffic.
    For example: 
    esxcli network  ip interface list
vmk0
   Name: vmk0
   MAC Address: e4:11:5b:11:8c:16
   Enabled: true
   Portset: vSwitch0
   Portgroup: Management Network
   Netstack Instance: defaultTcpipStack
   VDS Name: N/A
   VDS UUID: N/A
   VDS Port: N/A
   VDS Connection: -1
   Opaque Network ID: N/A
   Opaque Network Type: N/A
   External ID: N/A
   MTU: 1500
   TSO MSS: 65535
   Port ID: 33554437

vmk1
   Name: vmk1
   MAC Address: 00:50:56:6a:3a:74
   Enabled: true
   Portset: vSwitch1
   Portgroup: vsandata
   Netstack Instance: defaultTcpipStack
  VDS Name: N/A
   VDS UUID: N/A
   VDS Port: N/A
   VDS Connection: -1
   Opaque Network ID: N/A
   Opaque Network Type: N/A
   External ID: N/A
   MTU: 9000
   TSO MSS: 65535
   Port ID: 50331660
    Note: Multicast information is included for backward compatibility. vSAN 6.6 and later releases do not require multicast.
  3. Use the esxcli vsan network ip add command to configure the management VMkernel network adapter to support witness traffic. 
    esxcli vsan network ip add -i vmkx -T witness
  4. Use the esxcli vsan network list command to verify the new network configuration.
    For example: 
    esxcli vsan network list
Interface
   VmkNic Name: vmk0
   IP Protocol: IP
   Interface UUID: 8cf3ec57-c9ea-148b-56e1-a0369f56dcc0
   Agent Group Multicast Address: 224.2.3.4
   Agent Group IPv6 Multicast Address: ff19::2:3:4
   Agent Group Multicast Port: 23451
   Master Group Multicast Address: 224.1.2.3
   Master Group IPv6 Multicast Address: ff19::1:2:3
   Master Group Multicast Port: 12345
   Host Unicast Channel Bound Port: 12321
   Multicast TTL: 5
   Traffic Type: witness

Interface
   VmkNic Name: vmk1
   IP Protocol: IP
   Interface UUID: 6df3ec57-4fb6-5722-da3d-a0369f56dcc0
   Agent Group Multicast Address: 224.2.3.4
   Agent Group IPv6 Multicast Address: ff19::2:3:4
   Agent Group Multicast Port: 23451
   Master Group Multicast Address: 224.1.2.3
   Master Group IPv6 Multicast Address: ff19::1:2:3
   Master Group Multicast Port: 12345
   Host Unicast Channel Bound Port: 12321
   Multicast TTL: 5
   Traffic Type: vsan

Results

In the vSphere Client, the management VMkernel network interface is not selected for vSAN traffic. Do not re-enable the interface in the vSphere Client.