Persistent Volume Expansion for TKG Service Clusters

You can use the Kubernetes volume expansion feature to expand a persistent block volume after its creation. TKG Service clusters support offline and online volume expansion.

About Persistent Volume Expansion

Storage classes that appear in the TKG cluster environment have allowVolumeExpansion set to true by default. This parameter makes it possible to modify the size of an offline or online volume.

A volume is considered to be offline when it is not attached to a node or pod. An online volume is a volume that is available on a node or pod.

The level of support of the volume expansion functionality depends on the vSphere version. You can expand volumes created in the earlier versions of vSphere when you upgrade your vSphere environment to appropriate versions that support expansions.

Note: You can expand only persistent block volumes. Currently, vSphere IaaS control plane does not support volume expansion for ReadWriteMany volumes.

When you expand a persistent block volume, keep in mind the following:

You can expand the volumes up to the limits specified by storage quotas. vSphere IaaS control plane supports consecutive resize requests for a persistent volume claim object.
All types of datastores, including VMFS, vSAN, vSAN Direct, vVols, and NFS, support volume expansion.
You can perform volume expansion for deployments or standalone pods.
You can resize statically provisioned volumes in a Tanzu Kubernetes Grid cluster if the volumes have storage classes associated with them.
You cannot expand volumes created as part of a StatefulSet.
If a virtual disk that backs a volume has snapshots, it cannot be resized.
vSphere IaaS control plane does not support volume expansion for in-tree or migrated volumes.

Expand a Persistent Volume in Online Mode

An online volume is a volume that is available on a node or pod. As a DevOps engineer, you can expand an online persistent block volume. Tanzu Kubernetes Grid clusters support online volume expansion.

Find the persistent volume claim to resize using the following command.

Note that in this example the size of storage the volume uses is 1 Gi.

$ kubectl get pv,pvc,pod
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM               STORAGECLASS   REASON   AGE
persistentvolume/pvc-5cd51b05-245a-4610-8af4-f07e77fdc984   1Gi        RWO            Delete           Bound       default/block-pvc   block-sc                4m56s
 
NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/block-pvc   Bound    pvc-5cd51b05-245a-4610-8af4-f07e77fdc984   1Gi        RWO            block-sc       5m3s
 
NAME            READY   STATUS    RESTARTS   AGE
pod/block-pod   1/1     Running   0          26s

Patch the PVC to increase its size. For example, increase the size to 2 Gi.
This action triggers an expansion in the volume associated with the PVC.
```
$ kubectl patch pvc block-pvc -p '{"spec": {"resources": {"requests": {"storage": "2Gi"}}}}'
persistentvolumeclaim/block-pvc edited
```

Verify that the size of both PVC and PV has increased.

$ kubectl get pvc,pv,pod
NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/block-pvc   Bound    pvc-5cd51b05-245a-4610-8af4-f07e77fdc984   2Gi        RWO            block-sc       6m18s
 
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM               STORAGECLASS   REASON   AGE
persistentvolume/pvc-5cd51b05-245a-4610-8af4-f07e77fdc984   2Gi        RWO            Delete           Bound       default/block-pvc   block-sc                6m11s
 
NAME            READY   STATUS    RESTARTS   AGE
pod/block-pod   1/1     Running   0          101s

Use the vSphere Client to verify the new peristent volume size.
See Monitor Volume Health in a Tanzu Kubernetes Grid Cluster.

Expand a Persistent Volume in Offline Mode

A volume is considered to be offline when it is not attached to a node or pod. Tanzu Kubernetes Grid clusters support offline volume expansion.

Create a persistent volume claim (PVC) for an existing storage class.

In the example, the size of the requested storage is 1 Gi.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: example-block-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: example-block-sc

kubectl apply -f example-block-pvc.yaml

Patch the PVC to increase its size.
If the PVC is not attached to a node or being used by a pod, use the following command to patch the PVC. In this example, the requested storage increase is 2 Gi.
This action triggers an expansion in the volume associated with the PVC.
```
kubectl patch pvc example-block-pvc -p '{"spec": {"resources": {"requests": {"storage": "2Gi"}}}}'
```

Verify that the size of the volume has increased.

kubectl get pv
NAME                                       CAPACITY ACCESS MODES RECLAIM POLICY STATUS   CLAIM                       STORAGECLASS           REASON AGE
pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1   2Gi           RWO         Delete     Bound    default/example-block-pvc   example-block-sc              6m44s

Verify that the resize of the PVC is pending.
Note: The size of the PVC remains unchanged until the PVC is used by a pod.
The following example shows that the PVC size hasn't changed because it has not been used by a pod. If you run kubectl describe pvc, you see the FilesystemResizePending condition applied on the PVC. Once it is used by a pod, it will change size.
```
kubectl get pvc
NAME                STATUS VOLUME                                     CAPACITY ACCESS MODES   STORAGECLASS       AGE
example-block-pvc   Bound  pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1   1Gi           RWO       example-block-sc   6m57s
```
Create a pod to use the PVC.
When the PVC is used by the pod, the filesystem is expanded.

Verify that the size of the PVC has been modified.

kubectl get pvc
NAME                STATUS VOLUME                                    CAPACITY ACCESS MODES STORAGECLASS     AGE
example-block-pvc   Bound  pvc-24114458-9753-428e-9c90-9f568cb25788   2Gi         RWO      example-block-sc 2m12s

The FilesystemResizePending condition has been removed from the PVC. Volume expansion is complete.

Use the vSphere Client to verify the new peristent volume size.
See Monitor Volume Health in a Tanzu Kubernetes Grid Cluster.