Deploy Workloads on a Preferential Datastore in a Topology-Aware Environment

vSphere Container Storage Plug-in supports volume provisioning on a preferential datastore. Preferential datastores are only supported for block volumes.

You can use this functionality in an environment with a replicated datastore that is shared across two topology domains, or sites. The datastore is active in one of the topology domains, and is passive in the other.

When a site failure occurs, the active datastore on the failed site becomes passive, and the passive datastore on the other site becomes active.

In the following diagram, the DS-1 datastore is active in Site 1 and passive in Site 2. The DS-2 datastore is active in Site 2 and passive in Site 1.

Both datastores, DS-1 and DS-2, are accessible to all nodes in both sites. A typical volume provisioning request for Site 1 would provision a volume on either DS-1 or DS-2.

You can set preference to a particular datastore for a site, so that volume provisioning is limited only to the active datastore.

In this example, the DS-1 datastore is set as a preferred datastore for Site 1 and DS-2 datastore is a preferred datastore for Site 2.

Prerequisites

• Ensure that the vSphere Container Storage Plug-in version is 2.6.1 or later.
• Preferential datastore requires a topology aware Kubernetes deployment. See Deploying vSphere Container Storage Plug-in with Topology.

Procedure

1. As a vSphere administrator, create a category with the following name:
   cns.vmware.topology-preferred-datastores.
2. Create tags under this category that match the tags that you used for the topology domain name. For example,
   • For Site 1, if the topology domain name is site-1, create a site-1 tag under the cns.vmware.topology-preferred-datastores category.
   • For Site 2, if the topology domain name is site-2, create a site-2 tag under the cns.vmware.topology-preferred-datastores category.
3. Assign the site-1 and site-2 tags created under the cns.vmware.topology-preferred-datastores category to the DS-1 and DS-2 datastore correspondingly.
   During a datastore failover scenario, workload node VMs created on a specific datastore become inaccessible. The API server objects for these nodes are deleted from the Kubernetes cluster by CPI. To fix this issue, it is recommended to install vSphere Cloud Provider v1.24.2 or later, if you are using Kubernetes version 1.24.

Results

After you set the preference on the datastores, any volume provisioning request for site-1 ensures that the volume is allocated on the DS-1 datastore.

What to do next

The CSI driver periodically acknowledges any modifications to preferred datastore configurations every 5 minutes. If required, you can expedite these changes by restarting the vSphere CSI controller pods.

To customize this interval, you can adjust or define the csi-fetch-preferred-datastores-intervalinmin setting within the global section of the vSphere config secret.