vSAN Witness Design for a Virtual Infrastructure Workload Domain

The vSAN witness appliance contains a special ESXi installation that provides quorum and tiebreaker services for stretched clusters in a VI workload domain of VMware Cloud Foundation.

vSAN Witness Deployment Specification

When using vSAN in a stretched cluster configuration, you must deploy a witness ESXi host on a physical server or as a virtual appliance. This appliance must be deployed in a third location that is not local to the ESXi hosts on either side of the stretched cluster.

Table 1. Resource Specification of a vSAN Witness Appliance
Appliance Size	Supported Capacity	Number of vCPUs	Memory	Storage
Tiny	Supports up to 10 virtual machines and 750 witness components	2	8 GB	The appliance has three virtual disks. ESXi boot disk: 12 GB HDD Caching device: 10 GB SSD Capacity device: 15 GB HDD
Medium	Supports up to 500 virtual machines and 21,000 witness components	2	16 GB	The appliance has three virtual disks. ESXi boot disk: 12 GB HDD Caching Device: 10 GB SSD Capacity Device : 350 GB HDD
Large	Supports over 500 virtual machines and 45,000 witness components	2	32 GB	The appliance has five virtual disks. ESXi boot disk: 12 GB HDD Caching device: 10 GB SSD Capacity devices: 3 x 350 GB HDD each

Table 2. Design Decisions for the vSAN Witness Appliance for Multiple Availability Zones
Decision ID	Design Decision	Design Justification	Design Implication
VCF-WLD-vSAN-WTN-001	Deploy a vSAN witness appliance in a location that is not local to the ESXi hosts in any of the availability zones of the VI workload domain.	The witness appliance has these features. Acts as a tiebreaker if network isolation between the availability zones occurs. Hosts all required witness components to form the required RAID-1 configuration on vSAN, that is, each data copy at a site while the witness is at the witness site.	A third physically-separate location is required. Such a location must have a vSphere environment running to host the witness appliance. Another VMware Cloud Foundation Instance in a separate physical location might be an option.
VCF-WLD-vSAN-WTN-002	Deploy a large-size witness appliance.	A large-size witness appliance supports more than 500 virtual machines which is required for high availability of workloads that run in the SDDC.	The vSphere environment at the witness location must satisfy the resource requirements of the witness appliance.

vSAN Witness Network Design

When using two availability zones, to be able to communicate to the vCenter Server instance, connect the vSAN witness appliance for the VI workload domain to a network that is routed to the management network of the management domain in the first availability zone.

VMware Cloud Foundation uses vSAN witness traffic separation where you can use a VMkernel adapter for vSAN witness traffic that is different from the adapter for vSAN data traffic. You configure vSAN witness traffic in the following way:

On each ESXi host in both availability zones, place the vSAN witness traffic on the management VMkernel adapter.
On the vSAN witness appliance, use the same VMkernel adapter for both management and witness traffic. This VMkernel adapter is connected to a network that is routed to the management networks of the management domain and the VI workload domain in both availability zones.

For information about vSAN witness traffic separation, see vSAN Stretched Cluster Guide on VMware Cloud Platform Tech Zone.

Management Network

Routed to the management networks of the management domain and the VI workload domain in both availability zones. Connect the first VMkernel adapter of the vSAN witness appliance to this network. The second VMkernel adapter on the vSAN witness appliance is not used.

Place the following traffic on this network:

Management traffic
To be able to communicate to the vCenter Server instance, the vSAN witness appliance for the VI workload domain must be routed to the management network in Availability Zone 1.
vSAN witness traffic

The witness appliance is connected to the management network in the third location for management and witness traffic. The management network is routed to the management networks in the two availability zones of the management and VI workload domains. — Figure 1. vSAN Witness Network Design

Table 3. Design Decisions on the Network Configuration of the vSAN Witness Appliance for Multiple Availability Zones
Decision ID	Design Decision	Design Justification	Design Implication
VCF-WLD-vSAN-WTN-003	Connect the first VMkernel adapter of the vSAN witness appliance to the management network in the witness site.	Connects the witness appliance to the vCenter server instance and ESXi hosts in the VI workload domain.	The management networks of both the management and VI workload domains in both availability zones must be routed to the management network in the witness site.
VCF-WLD-vSAN-WTN-004	Configure the vSAN witness appliance to use the first VMkernel adapter, that is the management Interface, for vSAN witness traffic.	Separates the witness traffic from the vSAN data traffic. Witness traffic separation provides the following benefits: Removes the requirement to have static routes from the vSAN networks in both availability zones to the witness site. Removes the requirement to have jumbo frames enabled on the path between both availability zones and the witness site because witness traffic can use a regular MTU size of 1500 bytes.	The management networks for both the management and VI workload domains in both availability zones must be routed to the management network in the witness site.
VCF-WLD-vSAN-WTN-005	Place witness traffic on the management VMkernel adapter of all the ESXi hosts in the VI workload domain.	Separates the witness traffic from the vSAN data traffic. Witness traffic separation provides the following benefits: Removes the requirement to have static routes from the vSAN networks in both availability zones to the witness site. Removes the requirement to have jumbo frames enabled on the path between both availability zones and the witness site because witness traffic can use a regular MTU size of 1500 bytes.	The management networks for both the management and VI workload domains in both availability zones must be routed to the management network in the witness site.
VCF-WLD-vSAN-WTN-006	Allocate a statically assigned IP address and host name to the management adapter of the vSAN witness appliance.	Simplifies maintenance and tracking and implements a DNS configuration.	Requires precise IP address management.
VCF-WLD-vSAN-WTN-007	Configure forward and reverse DNS records for the vSAN witness appliance assigning the record to the child domain for the VMware Cloud Foundation instance.	Enables connecting the vSAN witness appliance to the VI workload domain vCenter Server by FQDN instead of by IP address.	You must provide DNS records for the vSAN witness appliance.
VCF-WLD-vSAN-WTN-008	Configure time synchronization by using an internal NTP time for the vSAN witness appliance.	Prevents any failures in the stretched cluster configuration that are caused by time mismatch between the vSAN witness appliance and the ESXi hosts in both availability zones and VI workload domain vCenter Server.	An operational NTP service must be available in the environment. All firewalls between the vSAN witness appliance and the NTP servers must allow NTP traffic on the required network ports.