Create Workload Clusters

This topic explains how to create workload clusters with the Tanzu CLI when using Tanzu Kubernetes Grid with a management cluster.

Important
If you are using TKG with a vSphere IaaS control plane (formerly vSphere with Tanzu) Supervisor, see the vSphere IaaS control plane docs.

Prerequisites

You have followed the prerequisites in Prerequisites for Cluster Deployment.
You have reviewed Configuration Files and Object Specs and configured a workload cluster as described in vSphere with Standalone Management Cluster Configuration Files.

(Recommended) Create a Cluster from a Configuration File

The procedures below explain how to deploy a workload cluster from a cluster configuration file. From a cluster configuration file, you can create the following types of clusters:

Class-based clusters: Follow the steps in Create a Class-Based Cluster below if you are deploying your workload cluster to vSphere with a standalone management cluster. For more information about cluster configuration files and object specs, see Configuration Files.
(Legacy) Plan-based clusters: Follow the steps in (Legacy) Create a Plan-Based Cluster below.

For more information about these cluster types, see Workload Cluster Types in About Tanzu Kubernetes Grid.

Create a Class-Based Cluster

The procedure below explains how to deploy a class-based workload cluster from a cluster configuration file. The resulting cluster is represented by a Cluster object in Kubernetes.

Important
VMware recommends using and retaining a dedicated configuration file for every cluster that you deploy.

Locate the configuration file that you prepared as part of Prerequisites above.
Create the cluster. You can create a cluster in either one or two steps, depending on whether you want to examine or edit its object spec before the object is created:
- (Recommended) When you create a cluster in one step, you pass your cluster configuration file to the --file option of tanzu cluster create, and the command automatically applies it.
- When you create a cluster in two steps, you pass your cluster configuration file to the --file option of tanzu cluster create, and the command then converts the file into a Cluster object spec and exits without creating the cluster. After examining or editing the spec, you create the cluster by re-running tanzu cluster create.
One-step process (recommended)
To follow the one-step process:
1. Set auto-apply-generated-clusterclass-based-configuration to true if it is not already. This configures the Tanzu CLI to always create class-based clusters using the one-step process. For more information about auto-apply-generated-clusterclass-based-configuration, see Features in Tanzu CLI Architecture and Configuration.
  
  tanzu config set features.cluster.auto-apply-generated-clusterclass-based-configuration true
2. Run tanzu cluster create, specifying the path to the cluster configuration file in the --file option. For example, if you saved the configuration file my-workload-cluster.yaml in the default clusterconfigs folder, run the following command to create a cluster with a name that you specified in the configuration file:
  
  tanzu cluster create --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml
  
  If you did not specify a name in the cluster configuration file or you want to create a cluster with a different name to the one that you specified, specify the cluster name in the tanzu cluster create command. For example, to create a cluster named another-workload-cluster from the configuration file my-workload-cluster.yaml, run the following command:
  
  tanzu cluster create another-workload-cluster --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml
Two-step process, always
To follow the two-step process for every cluster that you create:
1. Set the auto-apply-generated-clusterclass-based-configuration feature to false if it is not already. This configures the Tanzu CLI to always create class-based clusters using the two-step process. false is the default setting. If you have changed the default setting, to set it back to false, run:
  
  tanzu config set features.cluster.auto-apply-generated-clusterclass-based-configuration false
  
  For more information about auto-apply-generated-clusterclass-based-configuration, see Features in Tanzu CLI Architecture and Configuration.
2. To generate the object spec, run tanzu cluster create, specifying the path to the cluster configuration file in the --file option. The command saves the resulting object spec to the ~/.config/tanzu/tkg/clusterconfigs folder, prints its location, and then exits.
  
  For example, if you saved the cluster configuration file my-workload-cluster.yaml in the default clusterconfigs folder, run the following command to generate the object spec:
  
  tanzu cluster create --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml
  
  If you did not specify a name for your cluster in the configuration file or you want to create a cluster with a different name to the one that you specified, specify the cluster name in the tanzu cluster create command. For example:
  
  tanzu cluster create another-workload-cluster --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml
3. Examine or edit the object spec file generated by tanzu cluster create.
4. Re-run tanzu cluster create, specifying the path to the object spec in the --file option. For example:
  
  tanzu cluster create --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster-spec.yaml
  
  Include all the same flags that you used in the first step, including the --tkr flag if you are creating a cluster that runs a different Kubernetes version than the management cluster. For example:
  
  tanzu cluster create --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster-spec.yaml -v 6 --tkr v1.26.14---vmware.1-tkg.1
Two-step process, one time
To follow the two-step process for a single cluster:
1. To generate the object spec, run tanzu cluster create with the --dry-run option. The --dry-run option overrides the auto-apply-generated-clusterclass-based-configuration setting.
  
  tanzu cluster create CLUSTER-NAME --dry-run --file PATH-TO-CLUSTER-CONFIG-FILE.yaml > PATH-TO-OBJECT-SPEC-FILE.yaml
  
  Where:
  
  CLUSTER-NAME is the name of the cluster. You can omit CLUSTER-NAME if you specified it in the cluster configuration file.
  
  PATH-TO-CLUSTER-CONFIG-FILE is the path to the cluster configuration file that you located in step 1.
  
  PATH-TO-OBJECT-SPEC-FILE is the location to which you want to save the resulting object spec file.
  
  For example, to save the resulting object spec to a file named my-workload-cluster-spec.yaml, run:
  
  tanzu cluster create my-cluster --dry-run --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml > my-workload-cluster-spec.yaml
2. Examine or edit the object spec file generated by the --dry-run option in the previous step. In the example above, the name of the spec file is my-workload-cluster-spec.yaml.
3. After you examine or edit your object spec file, re-run tanzu cluster create without the --dry-run option. In the --file option, specify the path to the object spec file. For example:
  
  tanzu cluster create my-cluster --file my-workload-cluster-spec.yaml
  
  Include all the same flags that you used in the first step, including the --tkr flag if you are creating a cluster that runs a different Kubernetes version than the management cluster. For example:
  
  tanzu cluster create --file my-workload-cluster-spec.yaml -v 6 --tkr v1.26.14---vmware.1-tkg.1
Note
When creating class-based clusters, the Tanzu CLI does not use ytt customizations described in Legacy Cluster Configuration with ytt. If the CLI detects them on your machine, it outputs an error It seems like you have done some customizations to the template overlays.
After the cluster has been created, run the tanzu cluster get command to see information about the cluster:
```
tanzu cluster get CLUSTER-NAME
```
The output lists information about the status of the control plane and worker nodes, the Kubernetes version that the cluster is running, and the names of the nodes.

(Legacy) Create a Plan-Based Cluster

The procedure below explains how to deploy a plan-based from a configuration file. The resulting cluster is represented by a VSphereCluster object in Kubernetes.

To create the cluster:

Set the allow-legacy-cluster feature to true in the Tanzu CLI:

tanzu config set features.cluster.allow-legacy-cluster true

Create the cluster:
1. Locate the configuration file that you prepared as part of Prerequisites above.
2. Run the tanzu cluster create command, specifying the path to the configuration file in the --file option. For example, if you saved the workload configuration file my-workload-cluster.yaml in the default clusterconfigs folder, run the following command to create a cluster with a name that you specified in the configuration file:
```
tanzu cluster create --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml
```
  If you did not specify a name in the configuration file or you want to create a cluster with a different name to the one that you specified, specify the cluster name in the tanzu cluster create command. For example, to create a cluster named another-workload-cluster from the configuration file my-workload-cluster.yaml, run the following command:
```
tanzu cluster create another-workload-cluster --file ~/.config/tanzu/tkg/clusterconfigs/my-workload-cluster.yaml
```
3. After the cluster has been created, run the tanzu cluster get command to see information about the cluster:
```
tanzu cluster get CLUSTER-NAME
```
  The output lists information about the status of the control plane and worker nodes, the Kubernetes version that the cluster is running, and the names of the nodes.

Create a Cluster from an Object Spec

The procedures below explain how to create a class-based workload cluster using a Kubernetes-style object spec:

Create a cluster object spec as described in Create an Object Spec below.
Deploy the cluster from the object spec as described in Create a Class-Based Cluster from the Object Spec below.

Create an Object Spec

To create a Kubernetes-style object spec file for a class-based workload cluster, follow the steps below.

If you are deploying the workload cluster to vSphere with a standalone management cluster, you can use the Tanzu CLI to convert a cluster configuration file into a Kubernetes-style object spec file for a class-based workload cluster without deploying the cluster. You can then use the spec as an example to create your own specs. To create a spec file, you can:

If you have updated the default configuration of the auto-apply-generated-clusterclass-based-configuration feature, set it back to false and then run tanzu cluster create with the --file flag. To set auto-apply-generated-clusterclass-based-configuration to false:

tanzu config set features.cluster.auto-apply-generated-clusterclass-based-configuration false

When this feature is set to false and you run tanzu cluster create with the --file flag, the command converts your cluster configuration file into an object spec file and exits without creating the cluster. After reviewing the configuration, you re-run tanzu cluster create with the object spec file generated by the Tanzu CLI.

To create the spec file for a single cluster, pass the --dry-run option to tanzu cluster create and save the output to a file. Use the same options and configuration --file that you would use if you were creating the cluster, for example:
```
tanzu cluster create my-cluster --file my-cluster-config.yaml --dry-run > my-cluster-spec.yaml
```
The --dry-run option overrides the auto-apply-generated-clusterclass-based-configuration setting.

For an example object spec file, see Example Cluster Object and Its Subordinate Objects.

Create a Class-Based Cluster from the Object Spec

To deploy a class-based workload cluster from an object spec, pass the object spec to the --file option of tanzu cluster create, for example:

tanzu cluster create my-cluster --file my-cluster-spec.yaml

Note
When creating class-based clusters, the Tanzu CLI does not use ytt customizations described in Legacy Cluster Configuration with ytt. If the CLI detects them on your machine, it outputs an error It seems like you have done some customizations to the template overlays.

Create a Shared Services Cluster

For workload clusters managed by a management cluster created with tanzu management-cluster create or tanzu mc create, deploying Harbor or other services enables all of the workload clusters to share a single service instance.

Each Tanzu Kubernetes Grid instance can have only one shared services cluster.

Deploying Harbor to a shared services cluster enables all workload clusters that are managed by the same management cluster to share a single Harbor instance. For instructions on deploying Harbor, See Install Harbor for Service Registry.

To create a shared services cluster:

Create a cluster configuration YAML file for the cluster. We recommend using the prod cluster plan rather than the dev plan. For example:
```
INFRASTRUCTURE_PROVIDER: vsphere
CLUSTER_NAME: YOUR-CLUSTER-NAME
CLUSTER_PLAN: prod
```
Where YOUR-CLUSTER-NAME is the name you choose for the cluster. For example, tkg-services.
If you are using the default Kube-Vip load balancer for cluster’s control plane API, you must specify its endpoint by setting VSPHERE_CONTROL_PLANE_ENDPOINT. Ensure that this VIP address is not in the DHCP range, but is in the same subnet as the DHCP range. If you mapped a fully qualified domain name (FQDN) to the VIP address, you can specify the FQDN instead of the VIP address.
If you are using NSX Advanced Load Balancer (ALB), do not set VSPHERE_CONTROL_PLANE_ENDPOINT unless you need the control plane endpoint to be specific address. If so, use a static address within the NSX ALB IPAM Profile’s VIP Network range that you have manually added to the Static IP pool, or an FQDN mapped to the static address.

For example:
```
VSPHERE_CONTROL_PLANE_ENDPOINT: 10.10.10.10
```
Create the shared services cluster as described in Create a Cluster from a Configuration File, above.
Set the context of kubectl to the context of your management cluster. For example:
```
kubectl config use-context mgmt-cluster-admin@mgmt-cluster
```
In this example, mgmt-cluster is the name of the management cluster.
Add the tanzu-services label to the shared services cluster, as its cluster role. This label identifies the shared services cluster to the management cluster and workload clusters. For example:
```
kubectl label cluster.cluster.x-k8s.io/tkg-services cluster-role.tkg.tanzu.vmware.com/tanzu-services="" --overwrite=true
```
In this example, tkg-services is the name of the shared services cluster. You should see the confirmation cluster.cluster.x-k8s.io/tkg-services labeled.

Check that the label has been correctly applied by running the following command:

tanzu cluster list --include-management-cluster

You should see that your shared services cluster has the tanzu-services role. For example:

NAME              NAMESPACE   STATUS   CONTROLPLANE  WORKERS  KUBERNETES         ROLES           PLAN   TKR
another-cluster   default     running  1/1           1/1      v1.28.11+vmware.1  <none>          dev    v1.28.11---vmware.1-tkg.1
tkg-services      default     running  3/3           3/3      v1.28.11+vmware.1  tanzu-services  prod   v1.28.11---vmware.1-tkg.1
mgmt-cluster      tkg-system  running  1/1           1/1      v1.28.11+vmware.1  management      dev    v1.28.11---vmware.1-tkg.1

Get the admin credentials of the shared services cluster. For example:
```
tanzu cluster kubeconfig get tkg-services --admin
```
Set the context of kubectl to the shared services cluster. For example:
```
kubectl config use-context tkg-services-admin@tkg-services
```

What to Do Next

Configure the IP addresses of the cluster control plane endpoint and its nodes to be static, as described in Configure DHCP Reservations for the Control Plane Endpoint and All Nodes.
To connect to a workload cluster, see Connect to and Examine Workload Clusters.