You can use the Kubernetes volume expansion feature to expand a persistent block volume after its creation. TKG Service clusters support offline and online volume expansion.
About Persistent Volume Expansion
Storage classes that appear in the TKG cluster environment have allowVolumeExpansion
set to true
by default. This parameter makes it possible to modify the size of an offline or online volume.
A volume is considered to be offline when it is not attached to a node or pod. An online volume is a volume that is available on a node or pod.
The level of support of the volume expansion functionality depends on the vSphere version. You can expand volumes created in the earlier versions of vSphere when you upgrade your vSphere environment to appropriate versions that support expansions.
- You can expand the volumes up to the limits specified by storage quotas. vSphere IaaS control plane supports consecutive resize requests for a persistent volume claim object.
- All types of datastores, including VMFS, vSAN, vSAN Direct, vVols, and NFS, support volume expansion.
- You can perform volume expansion for deployments or standalone pods.
- You can resize statically provisioned volumes in a TKG cluster if the volumes have storage classes associated with them.
- You cannot expand volumes created as part of a StatefulSet.
- If a virtual disk that backs a volume has snapshots, it cannot be resized.
- vSphere IaaS control plane does not support volume expansion for in-tree or migrated volumes.
Expand a Persistent Volume in Online Mode
An online volume is a volume that is available on a node or pod. As a DevOps engineer, you can expand an online persistent block volume. TKG clusters support online volume expansion.
- Find the persistent volume claim to resize using the following command.
Note that in this example the size of storage the volume uses is 1 Gi.
$ kubectl get pv,pvc,pod NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/pvc-5cd51b05-245a-4610-8af4-f07e77fdc984 1Gi RWO Delete Bound default/block-pvc block-sc 4m56s NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/block-pvc Bound pvc-5cd51b05-245a-4610-8af4-f07e77fdc984 1Gi RWO block-sc 5m3s NAME READY STATUS RESTARTS AGE pod/block-pod 1/1 Running 0 26s
- Patch the PVC to increase its size. For example, increase the size to 2 Gi.
This action triggers an expansion in the volume associated with the PVC.
$ kubectl patch pvc block-pvc -p '{"spec": {"resources": {"requests": {"storage": "2Gi"}}}}' persistentvolumeclaim/block-pvc edited
- Verify that the size of both PVC and PV has increased.
$ kubectl get pvc,pv,pod NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/block-pvc Bound pvc-5cd51b05-245a-4610-8af4-f07e77fdc984 2Gi RWO block-sc 6m18s NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/pvc-5cd51b05-245a-4610-8af4-f07e77fdc984 2Gi RWO Delete Bound default/block-pvc block-sc 6m11s NAME READY STATUS RESTARTS AGE pod/block-pod 1/1 Running 0 101s
- Use the vSphere Client to verify the new peristent volume size.
Expand a Persistent Volume in Offline Mode
A volume is considered to be offline when it is not attached to a node or pod. TKG clusters support offline volume expansion.
- Create a persistent volume claim (PVC) for an existing storage class.
In the example, the size of the requested storage is 1 Gi.
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: example-block-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: example-block-sc
kubectl apply -f example-block-pvc.yaml
- Patch the PVC to increase its size.
If the PVC is not attached to a node or being used by a pod, use the following command to patch the PVC. In this example, the requested storage increase is 2 Gi.
This action triggers an expansion in the volume associated with the PVC.kubectl patch pvc example-block-pvc -p '{"spec": {"resources": {"requests": {"storage": "2Gi"}}}}'
- Verify that the size of the volume has increased.
kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1 2Gi RWO Delete Bound default/example-block-pvc example-block-sc 6m44s
- Verify that the resize of the PVC is pending.
Note: The size of the PVC remains unchanged until the PVC is used by a pod.The following example shows that the PVC size hasn't changed because it has not been used by a pod. If you run
kubectl describe pvc
, you see theFilesystemResizePending
condition applied on the PVC. Once it is used by a pod, it will change size.kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE example-block-pvc Bound pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1 1Gi RWO example-block-sc 6m57s
- Create a pod to use the PVC.
When the PVC is used by the pod, the filesystem is expanded.
- Verify that the size of the PVC has been modified.
kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE example-block-pvc Bound pvc-24114458-9753-428e-9c90-9f568cb25788 2Gi RWO example-block-sc 2m12s
TheFilesystemResizePending
condition has been removed from the PVC. Volume expansion is complete. - Use the vSphere Client to verify the new peristent volume size.