Certain Kubernetes workloads require persistent storage to store data permanently. To provision persistent storage for Kubernetes workloads, vSphere with Tanzu integrates with Cloud Native Storage (CNS), a vCenter Server component that manages persistent volumes.

Persistent storage is used by vSphere Pods, Tanzu Kubernetes clusters, and VMs. The following example illustrates how persistent storage is used by a vSphere Pod.

vSphere with Tanzu integrates with Cloud Native Storage to provision persistent storage.

To understand how vSphere with Tanzu works with persistent storage, be familiar with the following essential concepts.

Persistent Volume

To provide persistent storage, Kubernetes uses persistent volumes that can retain their state and data. If persistent volumes are mounted by a pod, they continue to exist even when the pod is deleted or reconfigured. In the vSphere with Tanzu environment, the persistent volume objects are backed by the First Class Disks on a datastore.

vSphere with Tanzu supports dynamic and static provisioning of volumes in ReadWriteOnce mode, where the volumes can be mounted by a single pod.

Starting with vSphere 7.0 Update 3 release, vSphere with Tanzu also supports ReadWriteMany mode for persistent volumes in Tanzu Kubernetes clusters. With the ReadWriteMany support, a single volume can be mounted simultaneously by multiple pods or applications running in a cluster. vSphere with Tanzu uses vSAN file shares for persistent volumes of the ReadWriteMany type. For more information,see Creating ReadWriteMany Persistent Volumes in vSphere with Tanzu.

Dynamic and Static Provisioning

With dynamic volume provisioning, storage does not need to be pre-provisioned and persistent volumes can be created on demand. DevOps engineers issue a persistent volume claim that references a storage class available in the namespace. vSphere with Tanzu automatically provisions the corresponding persistent volume and a backing virtual disk.

Both the Supervisor Cluster and Tanzu Kubernetes cluster support dynamic provisioning.

For an example of how to dynamically create a persistent volume, see Provision a Dynamic Persistent Volume for a Stateful Application.

With static provisioning, you can use an existing storage object and make it available to a cluster.

Typically, a DevOps engineer must know the details of the existing storage object, its supported configurations, and mount options to be able to reuse it.

For an example of how to provision a static persistent volume, see Provision a Static Persistent Volume in a Tanzu Kubernetes Cluster.

First Class Disk

vSphere with Tanzu uses the First Class Disk (FCD) type of virtual disks to back persistent volumes. First Class Disk, also known as Improved Virtual Disk, is a named virtual disk not associated with a VM.

First Class Disks are identified by UUID. This UUID is globally unique and is the primary identifier for the FCD. The UUID remains valid even if its FCD is relocated or snapshotted.

Persistent Volume Claim

DevOps engineers create persistent volume claims to request persistent storage resources. The request provisions a persistent volume object and a matching virtual disk. In the vSphere Client, the persistent volume claim manifests as an FCD virtual disk that can be monitored by vSphere administrators.

The claim is bound to the persistent volume. The workloads can use the claim to mount the persistent volumes and access storage.

When the DevOps engineers delete the claim, the corresponding persistent volume object and the provisioned virtual disk are also deleted.

Storage Class

Kubernetes uses storage classes to describe requirements for storage backing the persistent volumes. DevOps engineers can include a specific storage class in their persistent volume claim specification to request the type of storage the class describes.

Persistent Storage Workflow

The workflow for provisioning persistent storage in vSphere with Tanzu generally includes the following sequential actions.

Step

Action

Description

1

vSphere administrators deliver persistent storage resources to the DevOps team.

vSphere administrators create VM storage policies that describe different storage requirements and classes of services. They can then assign the storage policies to a vSphere Namespace.

2

vSphere with Tanzu creates storage classes that match the storage policies assigned to the vSphere Namespace.

The storage classes automatically appear in the Kubernetes environment, and can be used by the DevOps team. If a vSphere administrator assigns multiple storage policies to the vSphere Namespace, a separate storage class is created for each storage policy.

If you use the Tanzu Kubernetes Grid Service to provision Tanzu Kubernetes clusters, each Tanzu Kubernetes cluster inherits storage classes from the vSphere Namespace in which the cluster is provisioned.

3

DevOps engineers use the storage classes to request persistent storage resources for a workload.

The request comes in a form of a persistent volume claim that references a specific storage class.

4

vSphere with Tanzu creates a persistent volume object and a matching persistent virtual disk for a workload.

vSphere with Tanzu places the virtual disk into the datastore that meets the requirements specified in the original storage policy and its matching storage class. The virtual disk can be mounted by a workload.

5

vSphere administrators monitor persistent volumes in the vSphere with Tanzu environment.

Using the vSphere Client, vSphere administrators monitor the persistent volumes and their backing virtual disks. They can also monitor storage compliance and health statuses of the persistent volumes.
DevOps engineers create PVCs to request persistent storage resources. The PVC references a specific storage class.

Watch this video to learn about the persistent storage in vSphere with Tanzu.

Storage Management Tasks of a vSphere Administrator

Generally, the persistent storage management tasks in vSphere with Tanzu include the following. As a vSphere administrator, you use the vSphere Client to perform these tasks.

  • Perform lifecycle operations for the VM storage policies.

    Before enabling a Supervisor Cluster and configuring namespaces, create storage policies for persistent storage. The storage policies are based on the storage requirements communicated to you by the DevOps engineers. See Create Storage Policies for vSphere with Tanzu.

    Note:

    Do not delete the storage policy from VMware vCenter or a vSphere Namespace when a persistent volume claim with the corresponding storage class is running in the namespace. This recommendation also applies to Tanzu Kubernetes clusters.

  • Provide storage resources to the DevOps engineers by assigning the storage policies to the namespace and by setting storage limits. For information about changing storage policy assignments, see Change Storage Settings on a Namespace. For information on setting limits, see Configure Object Limitations on a vSphere Namespace.

  • Monitor Kubernetes objects and their storage policy compliance in the vSphere Client. See Monitor Persistent Volumes in the vSphere Client.

Storage Management Tasks of a DevOps Engineer

Typically, the DevOps engineer uses kubectl to perform the following storage tasks.