vSphere with Tanzu supports persistent volumes in ReadWriteMany mode. With the ReadWriteMany support, a single volume can be mounted simultaneously by multiple pods or applications running in a TKG cluster. vSphere with Tanzu uses CNS file volumes backed by vSAN file shares for the ReadWriteMany persistent volumes. To use vSAN shares, you need to set up vSAN File Service in your vSAN environment and activate file volume support on your Supervisor.

Considerations for File Volumes

When you enable file volume support for persistent volumes in vSphere with Tanzu, keep in mind the following considerations.

  • File volumes are supported only for workloads in the Tanzu Kubernetes Grid cluster. They are not supported for workloads, such as vSphere Pods and VM Service VMs, in the Supervisor namespace.
  • When you request a RWX volume in Kubernetes, vSAN File Service creates an NFS based file share of the requested size and appropriate SPBM policy. One vSAN file share is created per a RWX volume. VMware supports 100 shares per vSAN File Service cluster, which means you can have no more than 100 RWX volumes.
  • With TKG clusters, use TKr version 1.22 or later.

    For information, see VMware Tanzu Kubernetes releases Release Notes.

  • When you enable file volume support for vSphere with Tanzu, be aware of the potential security weaknesses:
    • The volumes are mounted without encryption. The unencrypted data might be accessed while the data transits the network.
    • Access Control Lists (ACLs) is used for the file shares to isolate file share access within a supervisor namespace. It might have risk of IP spoofing.
  • Follow these guidelines for networking:
    • If you are using NSX for networking in vSphere with Tanzu, make sure that the Supervisor namespace has NAT mode enabled. See Create and Configure a vSphere Namespace on the Supervisor.
    • Make sure the vSAN File Service is routable from the Workload network and there is no NAT between the Workload network and vSAN File Service IP addresses.
    • Use common DNS server for vSAN File Service and vSphere with Tanzu.
  • If after enabling file volume support, you later deactivate it, existing ReadWriteMany persistent volumes that you provisioned in the cluster remain unaffected and usable. You will not be able to create new ReadWriteMany persistent volumes.
vSAN File Services and the TKG clusters

Workflow for Enabling File Volume Support for Persistent Volumes

Follow this process to enable file volume support.

  1. A vSphere administrator sets up a vSAN cluster with configured vSAN File Service. See Configure File Service.
  2. A vSphere administrator activates file volume support on the Supervisor. See Change Storage Settings on the Supervisor in the Installing and Configuring vSphere with Tanzu documentation.
  3. A DevOps engineer provisions a persistent volume setting the PVC accessMode as ReadWriteMany.

    Several pods can be provisioned with the same PVC.

    For example:
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-pvc
    spec:
      accessModes:
        - ReadWriteMany
      storageClassName: gold
      resources:
        requests:
            storage: 3Gi