Using vSAN Data Persistence Platform with Modern Stateful Services

In vSphere IaaS control plane, you can use the vSAN Data Persistence platform for modern stateful services that require persistent storage. The platform provides a framework that enables third parties to integrate their service applications with underlying vSphere infrastructure.

About vSAN Data Persistence Platform

The benefits of using vSAN Data Persistence include the following:

Automatic Service Deployment and Scaling

Using the vSphere Client, administrators can install and deploy a modern stateful service on a Supervisor and grant access to the service namespace to DevOps engineers. The DevOps engineers can provision and scale instances of the stateful service dynamically in a self-service manner through Kubernetes APIs.

Service Monitoring Integrated with vCenter Server

Partners can build dashboard plugins that integrate with vCenter Server. Using the UI plugins, the vSphere administrators can manage and monitor the stateful services. In addition, vSAN offers health and capacity monitoring capabilities for these integrated third-party services.

Optimized Storage Configuration with vSAN Direct

vSAN Direct enables modern stateful services to interface directly with the underlying direct-attached storage for optimized I/O and storage efficiency.

The platform supports the following types of services:

• Object storage, such as MinIO.
• NoSQL databases, also called non-relational databases.
• Traditional databases.

vSphere Shared Nothing Storage

Most modern stateful services have a Shared Nothing Architecture (SNA). They consume non-replicated local storage and offer their own storage replication, compression, and other data operations. As a result, the services do not benefit when the same operations are performed by the underlying storage.

To avoid duplicating the operations, the vSAN Data Persistence platform offers two vSAN solutions with optimized data paths. The persistent service can run either on vSAN with the SNA storage policy or on a mostly raw local storage called vSAN Direct.

vSAN with SNA Storage Policy

With this technology, you can use a distributed replicated vSAN datastore with the vSAN host-local SNA policy. As a result, the SNA service application can control placement and take over the duty of maintaining data availability. The technology makes it easy for the persistent service to co-locate its compute instance and a storage object on the same physical ESXi host. With the host-local placement, it is possible to perform such operations as replication at the service layer and not at the storage layer.

The compute instance, such as a pod, comes up first on one of the nodes in the vSAN cluster. And then the vSAN object created with the vSAN SNA policy automatically has all its data placed on the same node where the pod is running.

The following example illustrates storage deployment of an application that uses the SNA storage class for its persistent volume. vSAN can select any disk group on the node for the persistent volume placement.

Total Copies of Data = 3

Expected Fault Tolerance = 2

Actual Failures Guaranteed to Be Tolerated = 2

vSAN Direct

Even though vSAN with the SNA storage policy can place data locally to the compute instance, an overhead exists of a distributed vSAN data path between the application and the physical storage device. With vSAN Direct, the stateful services applications can access mostly raw non-vSAN local storage through a more direct data path, which offers the most performance optimized solution.

With vSAN Direct, the vSphere administrator can claim host-local devices, and then manage and monitor the devices. vSAN Direct provides insights into the device health, performance, and capacity. On every local device it claims, vSAN Direct creates an independent VMFS datastore and makes it available as a placement choice to the application. The VMFS datastores that vSAN Direct manages are exposed as storage pools in Kubernetes. In the vSphere Client, they appear as vSAN Direct datastores.

The following illustrates persistent volumes placed locally on vSAN Direct disks.

When to Use vSAN with SNA or vSAN Direct

Follow these general recommendations when deciding which type of vSAN to use.

• Use vSAN with SNA when you want the cloud-native stateful application to share the physical infrastructure with other regular VMs or Kubernetes workloads. Each workload can define its own storage policy and can get the best of both worlds from a single cluster.
• Use vSAN Direct if you are creating a dedicated hardware cluster for the shared nothing cloud-native services.

vSAN Data Persistence Platform Operator

The vSAN Data Persistence platform (vDPP) operator is a component responsible for running and managing partner stateful services integrated with vSphere. The vDPP operator exposes available stateful services to the vSphere administrator. When the vSphere administrator enables a persistent service, for example, MinIO, the vDPP operator deploys an application-specific operator for the service on the Supervisor.

The application-specific operators are provided by the third party and must be vDPP compliant. The operator typically offers a CRD which provides a self-service interface for Kubernetes users to instantiate instances. vSphere IaaS control plane uses this operator and CRD to provision new service instances and manage and monitor them through the stateful services layer. Most of these operators use stateful sets for deploying their instances.

After the vSphere administrator enables a service, the following takes place.

• The vDPP operator activates a service-specific operator.
• The service-specific operator registers the UI plug-in.
• Storage-optimized storage policies are created.

Configuration Limits for vSAN Data Persistence Platform

VMware provides configuration limits in the VMware Configuration Maximums tool.

vSAN Data Persistence Maximums	Limits
Maximum number of persistent volumes per vSAN Data Persistence platform	1000
Maximum number of persistent volumes per service instance on the vSAN Data Persistence platform	60 to 80