Node Pool Configuration for ClusterClass-Based Clusters

There are differences in behavior when using the tanzu cluster node-pool commands against a cluster that uses a ClusterClass definition as opposed to using other tanzu commands. The one thing that all of these APIs have in common when operating against a ClusterClass based cluster is the need to update the cluster, specifically the cluster topology.

tanzu cluster node-pool delete

The tanzu cluster node-pool delete command is the most straightforward change. The cluster resource is updated to remove the definition of the machine deployment with the name of the node pool that is passed to the delete API. The API does still prohibit the deletion of the final node pool in the cluster.

tanzu cluster node-pool delete tkg-wc-vsphere -n md–0

tanzu cluster node-pool list

The tanzu cluster node-pool list command retrieves the MachineDeployment resources in the same way as in the plan-based cluster case and matches these resources against the machine deployments in the cluster topology. This allows the tanzu cluster node-pool list command to return the same MachineDeployment resources with updated names, to match the machine deployments that are defined in the cluster topology.

tanzu cluster node-pool list tkg-wc-vsphere

  NAME  NAMESPACE  PHASE      REPLICAS  READY  UPDATED  UNAVAILABLE
  md-0  default    Ready      1         1      1        0

Properties on Node Pools in ClusterClass Based Clusters

Properties that can be updated on node pools in ClusterClass based clusters can be broken down into two types:

• Direct properties of the machine deployment topology resources, such as replica count, that modify the machine deployment’s topolgy directly.
• Variable overrides, that rely on the definition of variables at the ClusterClass level. When deploying a cluster based on a ClusterClass, the cluster definition must specify a number of these variables, which apply to all machine deployments by default. The machine deployment topology allows these variables to be overridden on a machine deployment basis.

The definitive source for properties that can be modified are the ClusterClass objects themselves. The TKG ClusterClass objects all define a variable, worker, that contains worker node VM properties. This includes instanceType and vmSize for AWS and Azure respectively. The vSphere version includes properties for diskGiB, memoryMiB, and numCPU. TKG with Supervisor defines top level variables for vmClass and storageClass as well as an array for nodePoolVolumes for customzing worker nodes. TKG on vSphere also defines a vcenter variable for updating properties related to the vCenter Server instance.

All TKG ClusterClass objects define a nodePoolLabels variable that specifies the node labels to apply on the worker nodes.

Important

The node-pool commands provided by the Tanzu CLI require the above variables to be defined on the ClusterClass to support creating and updating node pools. If you use a custom ClusterClass these variables are required as top-level variable definitions in the ClusterClass. It is still possible to get and delete node pools on custom ClusterClass based clusters using the Tanzu CLI. There is a special case where tanzu cluster node-pool create and tanzu cluster node-pool update will work for custom ClusterClass objects, specifically if the node pool definition passed to the Tanzu CLI specifies at most the replicas, az, name, workerClass, and tkrResolver variables. In this case the workerClass must match a workerClass definition in the custom ClusterClass.

Update an Existing Node Pool with tanzu cluster node-pool set

When the tanzu cluster node-pool set command is called using the name of an existing node pool, the update will only modify the replica count and labels of the named node pool, if provided. Any other provided options are ignored. Specifically the cluster’s topology is updated with a new replica count and a nodePoolLabels variable override if these values are provided.

Create a New Node Pool with tanzu cluster node-pool set

Creating node pools can follow one of two paths. You can provide the tanzu cluster node-pool set command a base machine deployment that is used as the starting point for the new node pool. A deep copy of the named base machine deployment acts as a basis on which the user-provided properties are layered. In general this means all of the variable overrides of the copied machine deployment are updated with new values that you provide. This is different from the case where a base machine deployment is not provided. In this case, variable overrides are copied from the global variable definitions defined in the cluster topology and then the user provided values are layered on those. Additionally when you do not provide a base machine deployment, you must provide a workerClass and tkrResolver definition. TKG ClusterClass objects have a tkg-worker class defined by default, and should be used unless you have added your own worker class definition. The tkrResolver is the value of the tkr-resolver annotation.

Create a New Node Pool By Using a Base Machine Deployment

To use a base machine deployment to create a new node pool, run the following command:

tanzu cluster node-pool set tkg-wc-vsphere -f /path/to/node-pool.yml –-base-machine-deployment md-0

With the following contents in the node-pool.yml file:

name: np-1
replicas: 1
nodeMachineType: t3.large

Create a New Node Pool without Using a Base Machine Deployment

Create a new node pool without using a base machine deployment, run the following command:

tanzu cluster node-pool set tkg-wc-vsphere -f /path/to/node-pool.yml

With the following contents in the node-pool.yml file:

name: np-1
replicas: 1
nodeMachineType: t3.large
workerClass: tkg-worker
tkrResolver: os-name=ubuntu,os-arch=amd64