The Cloud Native storage design includes design considerations for stateful workloads that require persistent storage provided by SAN storage. The vSAN design forms the basis for the Cloud Native Storage design.

In Kubernetes, a Volume is a directory on a disk that is accessible to the containers inside a pod. Kubernetes supports many types of volumes. The Cloud Native storage design focuses on the vSAN storage design required to support dynamic volume provisioning. This design does not address different ways to present a volume to a stateful application.

vSAN Storage Policies:

The Telco Cloud Platform vSAN storage design provides the basis for container storage and has the following benefits:

  • Optimizes the storage design to meet the diverse needs of applications, services, administrators, and users.

  • Strategically aligns business applications and the storage infrastructure to reduce costs, boost performance, improve availability, provide security, and enhance functionality.

  • Provides multiple tiers of storage to match application data access to application requirements.

  • Designs each tier of storage with different performance, capacity, and availability characteristics. Not every application requires expensive, high-performance, highly available storage, so designing different storage tiers reduces cost.

vSAN storage policies define storage requirements for your StorageClass. Cloud Native persistent storage or volume (PV) inherits performance and availability characteristics made available by the vSAN storage policy. These policies determine how the storage objects are provisioned and allocated within the datastore to guarantee the required level of service. Kubernetes StorageClass is a way for Kubernetes admins to describe the “classes” of storage available for a Tanzu Kubernetes cluster by the Cloud Admin. Different StorageClasses map to different vSAN storage policies.

The following diagram is an example mapping of vSAN policy and Kubernetes StorageClass:

Figure 1. Cloud Native StorageClass Mapping to vSAN
Cloud Native StorageClass Mapping to vSAN
Note:

This design uses vSAN. Any supported storage solution that meets the characteristics of this storage design can be used. For best practices, see the vendor documentation.

Storage Access Modes

Cloud Native persistent storage or volume in Kubernetes is mounted with a certain access mode. Three possible access modes are as follows:

Access Mode

CLI Abbreviation

Description

ReadWriteOnce

RWO

The volume can be mounted as read-write by a single node.

ReadOnlyMany

ROX

The volume can be mounted read-only by many nodes.

ReadWriteMany

RWX

The volume can be mounted as read-write by many nodes.
RWO is a common access mode for cloud native Stateful workloads. RWO volumes have a 1:1 relation to a Pod in general. RWX volumes provide storage shared by multiple Pods with all Pods able to write to it. The difference between RWO and RWX relates to mounting the same filesystem on multiple hosts, which requires support for features such as distributed locking. With vSAN 7.0 and lower versions, the vSphere Cloud Storage Interface (CSI) based driver provisions only block-based Persistent Volume, which is RWO. RWX can be accomplished using external NFS. With vSAN 7.0 and higher versions, when the File Service feature is enabled, a single vSAN cluster can be used for both RWO and RWX support.

Cloud Native Storage Design Recommendations

Design Recommendation

Design Justification

Design Implication

Define a default StorageClass for all workloads in a Tanzu Kubernetes cluster.

Default StorageClass allows Kubernetes users that do not have strict storage requirements to consume persistent storage easier, without knowing the underlying implementation.

Performance-sensitive workloads might be incorrectly classified, if a Kubernetes user is left out of the StorageClass in a persistent volume claim.

Use the vSphere CSI provisioner for all RWO /RWX Persistent Volume Claims (PVC).

CSI provider is the out-of-tree storage provider implementation for vSphere. It offers advanced capabilities compared to in-tree vSphere provisioner.

Storage Provisioner is defined as part of StorageClass set manually.

provisioner: csi.vsphere.vmware.com

Create separate Storage-Classes based on access mode and storage type.

Note: The StorageClass default should be set to a Storageclass that maps to the most expected workload.

StorageClass is required to support dynamically provision storage for PersistentVolumeClaims.

Only a single default StorageClass is valid. If two or more storageClasses are marked as default, a PersistentVolumeClaim without storageClassName explicitly specified cannot be created.

Enable vSAN File Services for RWX PVC support.

File Services behave as NFS servers and provide access to the file shares required for PVC support.

Requires additional IP address management for File Service Agents.

Note: IP pool must be sized based on the number of vSAN Nodes.

When enabling vSAN File Services, create separate file shares for each Tanzu Kubernetes cluster. Enable Share warning threshold and quota based on cluster role or function.

  • Per cluster quota and file share simplify overall resource management.

  • Per Kubernetes cluster file share improves security.

None