The vSAN witness appliance contains a special ESXi installation that provides quorum and tiebreaker services for stretched clusters in the management domain of VMware Cloud Foundation.

vSAN Witness Deployment Specification

When using vSAN in a stretched cluster configuration, you must deploy a witness ESXi host on a physical server or as a virtual appliance. This appliance must be deployed in a third location that is not local to the ESXi hosts on either side of the stretched cluster.

Table 1. Resource Specification of a vSAN Witness Appliance

Appliance Size

Supported Capacity

Number of vCPUs

Memory

Storage

Tiny

Supports up to 10 virtual machines and 750 witness components

2

8 GB

The appliance has three virtual disks.

  • ESXi boot disk: 12 GB HDD

  • Caching device: 10 GB SSD

  • Capacity device: 15 GB HDD

Medium

Supports up to 500 virtual machines and 21,000 witness components

2

16 GB

The appliance has three virtual disks.

  • ESXi boot disk: 12 GB HDD

  • Caching Device: 10 GB SSD

  • Capacity Device : 350 GB HDD

Large

Supports over 500 virtual machines and 45,000 witness components

2

32 GB

The appliance has five virtual disks.

  • ESXi boot disk: 12 GB HDD

  • Caching device: 10 GB SSD

  • Capacity devices: 3 x 350 GB HDD each

Table 2. Design Decisions for the vSAN Witness Appliance for Multiple Availability Zones

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-WTN-001

Deploy a vSAN witness appliance in a location that is not local to the ESXi hosts in any of the availability zones.

The witness appliance has these features.

  • Acts as a tiebreaker if network isolation between the availability zones occurs.

  • Hosts all required witness components to form the required RAID-1 configuration on vSAN, that is, each data copy at a site while the witness is at the witness site.

A third physically-separate location is required. Such a location must have a vSphere environment. Another VMware Cloud Foundation Instance in a separate physical location might be an option.

VCF-MGMT-VSAN-WTN-002

Deploy a medium-size witness appliance.

A medium-size witness appliance supports up to 500 virtual machines which is sufficient for high availability of the management components of the SDDC.

The vSphere environment at the witness location must satisfy the resource requirements of the witness appliance.

vSAN Witness Network Design

When using two availability zones, connect the vSAN witness appliance to the management domain so that you can perform the initial setup of the stretched cluster in the management domain and have management workloads failed over between the zones.

VMware Cloud Foundation uses vSAN witness traffic separation where you can use a VMkernel adapter for vSAN witness traffic that is different from the adapter for vSAN data traffic. In this design, you configure vSAN witness traffic in the following way:

  • On each management ESXi host in both availability zones, place the vSAN witness traffic on the management VMkernel adapter.

  • On the vSAN witness appliance, use the same VMkernel adapter for both management and witness traffic.

For information about vSAN witness traffic separation, see vSAN Stretched Cluster Guide on VMware Cloud Platform Tech Zone.

Management network

Routed to the management networks in both availability zones. Connect the first VMkernel adapter of the vSAN witness appliance to this network. The second VMkernel adapter on the vSAN witness appliance is not used.

Place the following traffic on this network:

  • Management traffic

    To be able to communicate to the vCenter Server instance, the vSAN witness appliance for the management domain must access the management network in the first availability zone.

  • vSAN witness traffic

Figure 1. vSAN Witness Network Design

The vSAN witness appliance is connected to the management network in the third, witness, location using it to carry also witness traffic. The appliance is also connected to the vSAN storage in the witness location. The management network is routed to the management networks in the two availability zones of the management domain.
Table 3. Design Decisions on the Network Configuration of the vSAN Witness Appliance for Multiple Availability Zones

Decision ID

Design Decision

Design Justification

Design Implication

VCF-MGMT-VSAN-WTN-003

Connect the first VMkernel adapter of the vSAN witness appliance to the management network in the witness site.

Connects the witness appliance to the vCenter Server instance and ESXi hosts in both availability zones.

The management networks in both availability zones must be routed to the management network in the witness site.

VCF-MGMT-VSAN-WTN-004

Configure the vSAN witness appliance to use the first VMkernel adapter, that is the management interface, for vSAN witness traffic.

Separates the witness traffic from the vSAN data traffic. Witness traffic separation provides the following benefits:

  • Removes the requirement to have static routes from the vSAN networks in both availability zones to the witness site.

  • Removes the requirement to have jumbo frames enabled on the path between both availability zones and the witness site because witness traffic can use a regular MTU size of 1500 bytes.

The management networks in both availability zones must be routed to the management network in the witness site.

VCF-MGMT-VSAN-WTN-005

Place witness traffic on the management VMkernel adapter of all the ESXi hosts in the management domain.

Separates the witness traffic from the vSAN data traffic. Witness traffic separation provides the following benefits:

  • Removes the requirement to have static routes from the vSAN networks in both availability zones to the witness site.

  • Removes the requirement to have jumbo frames enabled on the path between both availability zones and the witness site because witness traffic can use a regular MTU size of 1500 bytes.

The management networks in both availability zones must be routed to the management network in the witness site.

VCF-MGMT-VSAN-WTN-006

Allocate a statically assigned IP address and host name to the management adapter of the vSAN witness appliance.

Simplifies maintenance and tracking, and implements a DNS configuration.

Requires precise IP address management.

VCF-MGMT-VSAN-WTN-007

Configure forward and reverse DNS records for the vSAN witness appliance assigning the record to the child domain for the VMware Cloud Foundation instance.

Enables connecting the vSAN witness appliance to the management domain vCenter Server by FQDN instead of IP address.

You must provide DNS records for the vSAN witness appliance.

VCF-MGMT-VSAN-WTN-008

Configure time synchronization by using an internal NTP time for the vSAN witness appliance.

Prevents any failures in the stretched cluster configuration that are caused by time mismatch between the vSAN witness appliance and the ESXi hosts in both availability zones and management domain vCenter Server.

  • An operational NTP service must be available in the environment.

  • All firewalls between the vSAN witness appliance and the NTP servers must allow NTP traffic on the required network ports.