Using vSAN File Service to Create ReadWriteMany Volumes in vSphere IaaS Control Plane

vSphere IaaS control plane supports persistent volumes in ReadWriteMany mode. With the ReadWriteMany support, a single volume can be mounted simultaneously by multiple pods or applications running in a TKG cluster. vSphere IaaS control plane uses CNS file volumes backed by vSAN file shares for the ReadWriteMany persistent volumes. To use vSAN shares, you need to set up vSAN File Service in your vSAN environment and activate file volume support on your Supervisor.

Considerations for File Volumes

When you enable file volume support for persistent volumes in vSphere IaaS control plane, keep in mind the following considerations.

• File volumes are supported only for workloads in the TKG cluster. They are not supported for workloads, such as vSphere Pods and VM Service VMs, in the Supervisor namespace.
• When you request a RWX volume in Kubernetes, vSAN File Service creates an NFS based file share of the requested size and appropriate SPBM policy. One vSAN file share is created per a RWX volume. VMware supports 100 shares per vSAN File Service cluster, which means you can have no more than 100 RWX volumes.
• With TKG clusters, use TKr version 1.22 or later.

  For information, see VMware Tanzu Kubernetes releases Release Notes.

• When you enable file volume support for vSphere IaaS control plane, be aware of the potential security weaknesses:
  • The volumes are mounted without encryption. The unencrypted data might be accessed while the data transits the network.
  • Access Control Lists (ACLs) is used for the file shares to isolate file share access within a supervisor namespace. It might have risk of IP spoofing.
• Follow these guidelines for networking:
  • If you are using NSX for networking in vSphere IaaS control plane, make sure that the Supervisor namespace has NAT mode enabled. See Create and Configure a vSphere Namespace on the Supervisor.
  • Make sure the vSAN File Service is routable from the Workload network and there is no NAT between the Workload network and vSAN File Service IP addresses.
  • Use common DNS server for vSAN File Service and vSphere IaaS control plane.
• If after enabling file volume support, you later deactivate it, existing ReadWriteMany persistent volumes that you provisioned in the cluster remain unaffected and usable. You will not be able to create new ReadWriteMany persistent volumes.

Workflow for Enabling File Volume Support for Persistent Volumes

Follow this process to enable file volume support.

1. A vSphere administrator sets up a vSAN cluster with configured vSAN File Service.
   • See Enable vSAN File Service and Configure File Service.
   • For specific settings in the environment with vSAN stretched cluster, see vSAN File Service with Stretched Cluster.
2. A vSphere administrator activates file volume support on the Supervisor.

   See Change Storage Settings on the Supervisor in the Installing and Configuring vSphere IaaS Control Plane documentation.

3. A DevOps engineer provisions a persistent volume setting the PVC accessMode as ReadWriteMany.

   Several pods can be provisioned with the same PVC.

   For example:

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc
   spec:
     accessModes:
       - ReadWriteMany
     storageClassName: gold
     resources:
       requests:
           storage: 3Gi