What Is vSAN

VMware vSAN is a distributed layer of software that runs natively as a part of the ESXi hypervisor.

vSAN aggregates local or direct-attached capacity devices of a host cluster and creates a single storage pool shared across all hosts in the vSAN cluster. While supporting VMware features that require shared storage, such as HA, vMotion, and DRS, vSAN eliminates the need for external shared storage and simplifies storage configuration and virtual machine provisioning activities.

vSAN Concepts

VMware vSAN uses a software-defined approach that creates shared storage for virtual machines.

It virtualizes the local physical storage resources of ESXi hosts and turns them into pools of storage that can be divided and assigned to virtual machines and applications according to their quality-of-service requirements. vSAN is implemented directly in the ESXi hypervisor.

You can configure vSAN to work as either a hybrid or all-flash cluster. In hybrid clusters, flash devices are used for the cache layer and magnetic disks are used for the storage capacity layer. In all-flash clusters, flash devices are used for both cache and capacity.

You can activate vSAN on existing host clusters, or when you create a new cluster. vSAN aggregates all local capacity devices into a single datastore shared by all hosts in the vSAN cluster. You can expand the datastore by adding capacity devices or hosts with capacity devices to the cluster. vSAN works best when all ESXi hosts in the cluster share similar or identical configurations across all cluster members, including similar or identical storage configurations. This consistent configuration balances virtual machine storage components across all devices and hosts in the cluster. Hosts without any local devices also can participate and run their virtual machines on the vSAN datastore.

In vSAN Original Storage Architecture (OSA), each host that contributes storage devices to the vSAN datastore must provide at least one device for flash cache and at least one device for capacity. The devices on the contributing host form one or more disk groups. Each disk group contains one flash cache device, and one or multiple capacity devices for persistent storage. Each host can be configured to use multiple disk groups.

In vSAN Express Storage Architecture (ESA), all storage devices claimed by vSAN contribute to capacity and performance. Each host's storage devices claimed by vSAN form a storage pool. The storage pool represents the amount of caching and capacity provided by the host to the vSAN datastore.

For best practices, capacity considerations, and general recommendations about designing and sizing a vSAN cluster, see the VMware vSAN Design and Sizing Guide.

Characteristics of vSAN

The following characteristics apply to vSAN, its clusters, and datastores.

vSAN includes numerous features to add resiliency and efficiency to your data computing and storage environment.

Table 1. vSAN Features
Supported Features	Description
Shared storage support	vSAN supports VMware features that require shared storage, such as HA, vMotion, and DRS. For example, if a host becomes overloaded, DRS can migrate virtual machines to other hosts in the cluster.
On-disk format	vSAN on-disk virtual file format provides highly scalable snapshot and clone management support per vSAN cluster. For information about the number of virtual machine snapshots and clones supported per vSAN cluster, refer to the vSphere Configuration Maximums https://configmax.esp.vmware.com/home.
All-flash and hybrid configurations	vSAN can be configured for all-flash or hybrid cluster.
Fault domains	vSAN supports configuring fault domains to protect hosts from rack or chassis failures when the vSAN cluster spans across multiple racks or blade server chassis in a data center.
File service	vSAN file service enables you to create file shares in the vSAN datastore that client workstations or VMs can access.
iSCSI target service	vSAN iSCSI target service enables hosts and physical workloads that reside outside the vSAN cluster to access the vSAN datastore.
vSAN Stretched cluster and Two node vSAN cluster	vSAN supports stretched clusters that span across two geographic locations.
Support for Windows Server Failover Clusters (WSFC)	vSAN 6.7 Update 3 and later releases support SCSI-3 Persistent Reservations (SCSI3-PR) on a virtual disk level required by Windows Server Failover Cluster (WSFC) to arbitrate an access to a shared disk between nodes. Support of SCSI-3 PRs enables configuration of WSFC with a disk resource shared between VMs natively on vSAN datastores. Currently the following configurations are supported: Up to 6 application nodes per cluster. Up to 64 shared virtual disks per node. Note: Microsoft SQL Server 2012 or later running on Microsoft Windows Server 2012 or later has been qualified on vSAN.
vSAN health service	vSAN health service includes preconfigured health check tests to monitor, troubleshoot, diagnose the cause of cluster component problems, and identify any potential risk.
vSAN performance service	vSAN performance service includes statistical charts used to monitor IOPS, throughput, latency, and congestion. You can monitor performance of a vSAN cluster, host, disk group, disk, and VMs.
Integration with vSphere storage features	vSAN integrates with vSphere data management features traditionally used with VMFS and NFS storage. These features include snapshots, linked clones, and vSphere Replication.
Virtual Machine Storage Policies	vSAN works with VM storage policies to support a VM-centric approach to storage management. If you do not assign a storage policy to the virtual machine during deployment, the vSAN Default Storage Policy is automatically assigned to the VM.
Rapid provisioning	vSAN enables rapid provisioning of storage in the vCenter Server^® during virtual machine creation and deployment operations.
Deduplication and compression	vSAN performs block-level deduplication and compression to save storage space. When you enable deduplication and compression on a vSAN all-flash cluster, redundant data within each disk group is reduced. Deduplication and compression is a cluster-wide setting, but the functions are applied on a disk group basis. Compression-only vSAN is applied on a per-disk basis.
Data at rest encryption	vSAN provides data at rest encryption. Data is encrypted after all other processing, such as deduplication, is performed. Data at rest encryption protects data on storage devices, in case a device is removed from the cluster.
Data in transit encryption	vSAN can encrypt data in transit across hosts in the cluster. When you enable data-in-transit encryption, vSAN encrypts all data and metadata traffic between hosts.
SDK support	The VMware vSAN SDK is an extension of the VMware vSphere Management SDK. It includes documentation, libraries and code examples that help developers automate installation, configuration, monitoring, and troubleshooting of vSAN.

vSAN Terms and Definitions

vSAN introduces specific terms and definitions that are important to understand.

Before you get started with vSAN, review the key vSAN terms and definitions.

Disk Group (vSAN Original Storage Architecture)

A disk group is a unit of physical storage capacity and performance on a host and a group of physical devices that provide performance and capacity to the vSAN cluster. On each ESXi host that contributes its local devices to a vSAN cluster, devices are organized into disk groups.

Each disk group must have one flash cache device and one or multiple capacity devices. The devices used for caching cannot be shared across disk groups, and cannot be used for other purposes. A single caching device must be dedicated to a single disk group. In hybrid clusters, flash devices are used for the cache layer and magnetic disks are used for the storage capacity layer. In an all-flash cluster, flash devices are used for both cache and capacity. For information about creating and managing disk groups, see Administering VMware vSAN.

Storage Pool (vSAN Express Storage Architecture)

A storage pool is a representation of all storage devices on a host that are claimed by vSAN. Each host contains one storage pool. Each device in the storage pool contributes both capacity and performance. The number of storage devices allowed is determined by the host configuration.

Consumed Capacity

Consumed capacity is the amount of physical capacity consumed by one or more virtual machines at any point. Many factors determine consumed capacity, including the consumed size of your .vmdk files, protection replicas, and so on. When calculating for cache sizing, do not consider the capacity used for protection replicas.

Object-Based Storage

vSAN stores and manages data in the form of flexible data containers called objects. An object is a logical volume that has its data and metadata distributed across the cluster. For example, every .vmdk is an object, as is every snapshot. When you provision a virtual machine on a vSAN datastore, vSAN creates a set of objects comprised of multiple components for each virtual disk. It also creates the VM home namespace, which is a container object that stores all metadata files of your virtual machine. Based on the assigned virtual machine storage policy, vSAN provisions and manages each object individually, which might also involve creating a RAID configuration for every object.

Note: If vSAN Express Storage Architecture is enabled, every snapshot is not a new object. A base .vmdk and its snapshots are contained in one vSAN object. Additionally, in vSAN ESA, digest is backed by vSAN objects.

When vSAN creates an object for a virtual disk and determines how to distribute the object in the cluster, it considers the following factors:

vSAN verifies that the virtual disk requirements are applied according to the specified virtual machine storage policy settings.
vSAN verifies that the correct cluster resources are used at the time of provisioning. For example, based on the protection policy, vSAN determines how many replicas to create. The performance policy determines the amount of flash read cache allocated for each replica and how many stripes to create for each replica and where to place them in the cluster.
vSAN continually monitors and reports the policy compliance status of the virtual disk. If you find any noncompliant policy status, you must troubleshoot and resolve the underlying problem.
Note: When required, you can edit VM storage policy settings. Changing the storage policy settings does not affect virtual machine access. vSAN actively throttles the storage and network resources used for reconfiguration to minimize the impact of object reconfiguration to normal workloads. When you change VM storage policy settings, vSAN might initiate an object recreation process and subsequent resynchronization. See vSAN Monitoring and Troubleshooting.
vSAN verifies that the required protection components, such as mirrors and witnesses, are placed on separate hosts or fault domains. For example, to rebuild components during a failure, vSAN looks for ESXi hosts that satisfy the placement rules where protection components of virtual machine objects must be placed on two different hosts, or across fault domains.

vSAN Datastore

After you enable vSAN on a cluster, a single vSAN datastore is created. It appears as another type of datastore in the list of datastores that might be available, including Virtual Volume, VMFS, and NFS. A single vSAN datastore can provide different service levels for each virtual machine or each virtual disk. In vCenter Server^®, storage characteristics of the vSAN datastore appear as a set of capabilities. You can reference these capabilities when defining a storage policy for virtual machines. When you later deploy virtual machines, vSAN uses this policy to place virtual machines in the optimal manner based on the requirements of each virtual machine. For general information about using storage policies, see the vSphere Storage documentation.

A vSAN datastore has specific characteristics to consider.

vSAN provides a single vSAN datastore accessible to all hosts in the cluster, whether or not they contribute storage to the cluster. Each host can also mount any other datastores, including Virtual Volumes, VMFS, or NFS.
You can use Storage vMotion to move virtual machines between vSAN datastores, NFS datastores, and VMFS datastores.
Only magnetic disks and flash devices used for capacity can contribute to the datastore capacity. The devices used for flash cache are not counted as part of the datastore.

Objects and Components

Each object is composed of a set of components, determined by capabilities that are in use in the VM Storage Policy. For example, with Failures to tolerate set to 1, vSAN ensures that the protection components, such as replicas and witnesses, are placed on separate hosts in the vSAN cluster, where each replica is an object component. In addition, in the same policy, if the Number of disk stripes per object configured to two or more, vSAN also stripes the object across multiple capacity devices and each stripe is considered a component of the specified object. When needed, vSAN might also break large objects into multiple components.

A vSAN datastore contains the following object types:

VM Home Namespace: The virtual machine home directory where all virtual machine configuration files are stored, such as .vmx, log files, .vmdk files, and snapshot delta description files.
VMDK: A virtual machine disk or .vmdk file that stores the contents of the virtual machine's hard disk drive.
VM Swap Object: Created when a virtual machine is powered on.
Snapshot Delta VMDKs: Created when virtual machine snapshots are taken. Such delta disks are not created for vSAN Express Storage Architecture.
Memory object: Created when the snapshot memory option is selected when creating or suspending a virtual machine.

Virtual Machine Compliance Status: Compliant and Noncompliant

A virtual machine is considered noncompliant when one or more of its objects fail to meet the requirements of its assigned storage policy. For example, the status might become noncompliant when one of the mirror copies is inaccessible. If your virtual machines are in compliance with the requirements defined in the storage policy, the status of your virtual machines is compliant. From the Physical Disk Placement tab on the Virtual Disks page, you can verify the virtual machine object compliance status. For information about troubleshooting a vSAN cluster, see vSAN Monitoring and Troubleshooting.

Component State: Degraded and Absent States

vSAN acknowledges the following failure states for components:

Degraded. A component is Degraded when vSAN detects a permanent component failure and determines that the failed component cannot recover to its original working state. As a result, vSAN starts to rebuild the degraded components immediately. This state might occur when a component is on a failed device.
Absent. A component is Absent when vSAN detects a temporary component failure where components, including all its data, might recover and return vSAN to its original state. This state might occur when you are restarting hosts or if you unplug a device from a vSAN host. vSAN starts to rebuild the components in absent status after waiting for 60 minutes.

Object State: Healthy and Unhealthy

Depending on the type and number of failures in the cluster, an object might be in one of the following states:

Healthy. When at least one full RAID 1 mirror is available, or the minimum required number of data segments are available, the object is considered healthy.
Unhealthy. An object is considered unhealthy when no full mirror is available or the minimum required number of data segments are unavailable for RAID 5 or RAID 6 objects. If fewer than 50 percent of an object's votes are available, the object is unhealthy. Multiple failures in the cluster can cause objects to become unhealthy. When the operational status of an object is considered unhealthy, it impacts the availability of the associated VM.

Witness

A witness is a component that contains only metadata and does not contain any actual application data. It serves as a tiebreaker when a decision must be made regarding the availability of the surviving datastore components, after a potential failure. A witness consumes approximately 2 MB of space for metadata on the vSAN datastore when using on-disk format 1.0, and 4 MB for on-disk format version 2.0 and later.

vSAN maintains a quorum by using an asymmetrical voting system where each component might have more than one vote to decide the availability of objects. Greater than 50 percent of the votes that make up a VM’s storage object must be accessible at all times for the object to be considered available. When 50 percent or fewer votes are accessible to all hosts, the object is no longer accessible to the vSAN datastore. Inaccessible objects can impact the availability of the associated VM.

Storage Policy-Based Management (SPBM)

When you use vSAN, you can define virtual machine storage requirements, such as performance and availability, in the form of a policy. vSAN ensures that the virtual machines deployed to vSAN datastores are assigned at least one virtual machine storage policy. When you know the storage requirements of your virtual machines, you can define storage policies and assign the policies to your virtual machines. If you do not apply a storage policy when deploying virtual machines, vSAN automatically assigns a default vSAN policy with Failures to tolerate set to 1, a single disk stripe for each object, and thin provisioned virtual disk. For best results, define your own virtual machine storage policies, even if the requirements of your policies are the same as those defined in the default storage policy. For information about working with vSAN storage policies, see Administering VMware vSAN.

vSphere PowerCLI

VMware vSphere PowerCLI adds command-line scripting support for vSAN, to help you automate configuration and management tasks. vSphere PowerCLI provides a Windows PowerShell interface to the vSphere API. PowerCLI includes cmdlets for administering vSAN components. For information about using vSphere PowerCLI, see vSphere PowerCLI Documentation.

How vSAN Differs from Traditional Storage

Although vSAN shares many characteristics with traditional storage arrays, the overall behavior and function of vSAN is different.

For example, vSAN can manage and work only with ESXi hosts, and a single vSAN instance provides a single datastore for the cluster.

vSAN and traditional storage also differ in the following key ways:

vSAN does not require external networked storage for storing virtual machine files remotely, such as on a Fibre Channel (FC) or Storage Area Network (SAN).
Using traditional storage, the storage administrator preallocates storage space on different storage systems. vSAN automatically turns the local physical storage resources of the ESXi hosts into a single pool of storage. These pools can be divided and assigned to virtual machines and applications according to their quality-of-service requirements.
vSAN does not behave like traditional storage volumes based on LUNs or NFS shares. The iSCSI target service uses LUNs to enable an initiator on a remote host to transport block-level data to a storage device in the vSAN cluster.
Some standard storage protocols, such as FCP, do not apply to vSAN.
vSAN is highly integrated with vSphere. You do not need dedicated plug-ins or a storage console for vSAN, compared to traditional storage. You can deploy, manage, and monitor vSAN by using the vSphere Client.
A dedicated storage administrator does not need to manage vSAN. Instead a vSphere administrator can manage a vSAN environment.
With vSAN, VM storage policies are automatically assigned when you deploy new VMs. The storage policies can be changed dynamically as needed.