Provisioning Volumes in an Environment with Multiple vCenter Server Instances

vSphere Container Storage Plug-in supports volume provisioning in an environment with multiple vCenter Server instances. As a result, you can provision volumes in a native Kubernetes cluster that spans across multiple vCenter Server instances.

In the following diagram, the Kubernetes cluster spans across three vCenter Server instances that represent different availability zones. Kubernetes control plane nodes and worker nodes are distributed across these three zones. A volume provisioning request with zone1, specified in a topology requirement, provisions the volume on vSAN1 or VMFS1 datastore connected to VC1.

The graphic shows a cluster that spans across three availability zones.

Requirements and Limitations

Before deploying a workload in the configuration with multiple vCenter Server instances, deploy vSphere Container Storage Plug-in with topology.

For more information, see Deploying vSphere Container Storage Plug-in with Multiple vCenter Server Instances.

Note: vSphere Container Storage Plug-in does not support datastores shared across multiple vCenter Server instances even if one of the vCenter Server instances does not use CNS or FCD.

Dynamic Volume Provisioning

To provision volumes and workloads dynamically in the environment with multiple vCenter Server instances, follow the same steps required for provisioning topology-aware volumes. For more information on topology-aware volume provisioning, see Topology-Aware Volume Provisioning.

The following additional guidelines apply.

In the storage class, specify topology segments, such as regions or zones, for only a single vCenter Server.
If topology segments in the storage class span across more than one vCenter Server, provisioning of a corresponding volume fails.

The following is an example of a storage class used for dynamic provisioning of a block or a file volume. Provisioning is done from one of the three vCenter Server instances shown in the diagram. In this example, the storage class indicates zone-1, which means that only VC-1 can initiate the provisioning.

kind: StorageClass

apiVersion: storage.k8s.io/v1

metadata:

     name: example-multi-zones-sc

     provisioner: csi.vsphere.vmware.com

allowedTopologies:

   - matchLabelExpressions:

         - key: topology.csi.vmware.com/k8s-zone

            values:

            - zone-1

Static Volume Provisioning

Static Volume Provisioning for File Volumes: To provision file volumes statically in an environment with multiple vCenter Server instances, follow the same steps required for provisioning static file volumes in a single vCenter Server. See Statically Provision File Volumes with vSphere Container Storage Plug-in.
The CSI driver can identify the vCenter Server where the backing file share is located and create a CNS volume for the same.

Static Volume Provisioning for Block Volumes

To provision block volumes statically in the environment with multiple vCenter Server instances, follow the same steps required for provisioning topology-aware volumes. For more information on topology-aware volume provisioning, see Topology-Aware Volume Provisioning.

Specify affinity rules for a PersistentVolume object in the nodeAffinity section. The affinity rules indicate the topology segment and vCenter Server, to which the volume belongs. If you do not specify the affinity rules the in PersistentVolume object, volume registration fails.

Use the following example, to provision a PersistentVolume object on VC-2 in the environment represented by the diagram.

apiVersion: v1

kind: PersistentVolume

metadata:

  name: static-pv-name

  annotations:

    pv.kubernetes.io/provisioned-by: csi.vsphere.vmware.com

  labels:

    static-pv-label-key: static-pv-label-value # This label is optional, it is used as a selector to bind with volume claim. This can be any unique key-value to identify PV.

spec:

  capacity:

    storage: 2Gi

  accessModes:

    - ReadWriteOnce

  persistentVolumeReclaimPolicy: Delete

  claimRef:

    namespace: default

    name: static-pvc-name

  csi:

    driver: "csi.vsphere.vmware.com"

    volumeAttributes:

      type: "vSphere CNS Block Volume"

    volumeHandle: "0c75d40e-7576-4fe7-8aaa-a92946e2805d" # First Class Disk (Improved Virtual Disk) ID

  nodeAffinity:

    required:

      nodeSelectorTerms:

        - matchExpressions:

            - key: topology.csi.vmware.com/k8s-zone

              operator: In

              values:

                - zone-2