This topic explains how to configure Tanzu Kubernetes Grid (TKG) workload clusters using a flat configuration file or a Kubernetes-style object spec. For IaaS-specific instructions, see:
How you configure a workload cluster depends on the cluster type, as described in the sections below.
(Default type) Class-based clusters use a Kubernetes-style object spec for the
Cluster object. To create a class-based cluster, you pass either this object spec or a cluster configuration file with a flat structure that sets uppercase-underscore variables like
tanzu cluster create.
ClusterBootstrapand objects that it references.
(Legacy types) Plan-based and TKC clusters use a cluster configuration file with a flat structure that sets uppercase-underscore variables like
CLUSTER_NAME. To create a legacy cluster, you pass this configuration file to
tanzu cluster create.
For more information about the above cluster types and the Cluster API providers that they use, see:
For information about which configuration file to choose for your workload cluster, see the table below.
|You can use a … with Tanzu CLI||to create workload clusters of type…||on infrastructure…|
|Flat cluster configuration file||
|Kubernetes-style object spec||Class-based clusters||vSphere 8 with Supervisor; vSphere 6.7, 7, and 8 with a standalone management cluster; AWS; and Azure|
You can use a cluster configuration file to configure the following types of clusters:
Class-based clusters: This includes deployments to:
If you want to deploy a class-based workload to vSphere 8 with Supervisor, you must deploy it from an object spec, as described in Configure a Supervisor-Deployed Class-Based Cluster (vSphere 8 only).
(Legacy) Plan-based and TKC clusters: This includes deployments to:
Before deploying a workload cluster to vSphere, AWS, or Azure, you create a configuration file for the cluster. When you pass this file to the
--file option of
tanzu cluster create, the Tanzu CLI uses the configuration information defined in the file to connect to your infrastructure provider and create the resources that the cluster will use.
To create a cluster configuration file, you can copy an existing configuration file for a previous deployment and update it. Alternatively, you can create a file from scratch by using an empty template:
To set the configuration information relevant to your infrastructure, see:
For the full list of options that you can specify in the cluster configuration file, see Configuration File Variable Reference.
When you deploy a workload cluster, most of the configuration for the cluster is the same as the configuration of the standalone management cluster that you use to deploy it. Because of this, the easiest way to create a configuration file for a workload cluster is to start with a copy of the standalone management cluster configuration file.
Locate the YAML configuration file for the management cluster:
--fileoption when you ran
tanzu mc create --ui, the configuration file is saved in
~/.config/tanzu/tkg/clusterconfigs/. The file has a randomly generated name, for example,
--fileoption, the management cluster configuration is taken from the file that you specified.
--fileoption, or from the default location,
Make a copy of the management cluster configuration file and save it with a new name. For example, save the file as
Update the settings in the cluster configuration file. If you want to configure a workload cluster to use an OS other than the default Ubuntu 20.04, you must set the
OS_VERSION values. The installer interface does not include node VM OS values in the management cluster configuration files that it saves to
Save the file.
To prepare a cluster configuration file for a TKC cluster on vSphere 8 with Supervisor, see Configure a Supervisor-Deployed TKC Cluster (Legacy).
You can use a Kubernetes-style object spec file to deploy a class-based workload cluster to:
An object spec file serves the same purpose as a cluster configuration file. When you pass an object spec to the
--file option of
tanzu cluster create, the Tanzu CLI creates the cluster using the configuration information defined in the object spec.
As with other Kubernetes objects, you can configure a class-based workload cluster by creating and editing object specs. For cluster objects, these are:
Clusterobject: Configures most cluster options, such as cluster topology. You can change and re-apply this object spec in order to change the configuration of a running cluster.
ClusterBootstrapobject: Configures non-default options that are only applied when the cluster is first created, such as the container network interface (CNI) and container storage interface (CSI), and which cannot be re-configured in a running cluster. For more information, see Configure One-Time Infrastructure Settings.
To create your first object spec file:
If you are deploying a class-based workload cluster to vSphere without Supervisor, AWS or Azure, you can use the Tanzu CLI to convert a cluster configuration file into an object spec file without deploying the cluster:
(Global configuration) Use the
auto-apply-generated-clusterclass-based-configuration feature of the Tanzu CLI. This feature is set to
false by default. When
auto-apply-generated-clusterclass-based-configuration is set to
false and you run
tanzu cluster create with the
--file flag, the command converts your cluster configuration file into an object spec file and exits without creating the cluster. After reviewing the configuration, you re-run
tanzu cluster create with the object spec file generated by the Tanzu CLI, as described in Create a Class-Based Cluster from the Object Spec. If you have updated the default configuration, to set it back to
tanzu config set features.cluster.auto-apply-generated-clusterclass-based-configuration false
(Single cluster) Pass the
--dry-run option to
tanzu cluster create and save the output to a file. Use the same options and configuration
--file that you would use if you were creating the cluster, for example:
tanzu cluster create my-cluster --file my-cluster-config.yaml --dry-run > my-cluster-spec.yaml
You can then use this object spec to deploy a cluster as described in Create a Class-Based Cluster from the Object Spec.
If you are deploying a class-based workload cluster to vSphere 8 with Supervisor, create or adapt a
Cluster object spec as described in Configuring a Supervisor-Deployed Class-Based Cluster (vSphere 8 only). The vSphere 8 documentation has example
Cluster object specs to work from, for example, v1beta1 Example: Default Cluster.
You can configure most common cluster settings in the
Cluster object spec, but some cluster components come from the Tanzu Kubernetes release (TKr) that the cluster’s nodes are based on. For example, the TKr specifies the CNI, CSI, and Pinniped versions that a cluster uses.
These infrastructure-level options are applied at cluster creation time by the
ClusterBootstrap object and cannot be changed in a running cluster. The general procedure for configuring these options before creating a cluster is:
Create object specs for the
ClusterBootstrap object and any custom objects that it references, for example, a
Create an object spec for the
Cluster object itself.
metadata for all of the object specs, include the name and namespace for the cluster that you are creating, for example:
apiVersion: run.tanzu.vmware.com/v1alpha3 kind: ClusterBootstrap metadata: name: MY-CLUSTER namespace: MY-NAMESPACE
The customization may require additional metadata, such as annotations.
Concatenate all of the object specs, including the
Cluster definition, into a single file. Separate the specs with lines consisting of three dashes (
Pass the object spec to the
--file option of
kubectl apply, for example:
kubectl config use-context CONTEXT-NAME
kubectl apply -f my-custom-cluster-objects.yaml
Alternative method: As an alternative to the last two steps, you can run
kubectl apply -f on all of the object specs except for the
Cluster object and then run
tanzu cluster create --file with the
Cluster object spec.
For a specific example of this procedure, see Create a Cluster with a Non-Default CNI, which sets a cluster’s CNI to Calico.
Tanzu Kubernetes Grid does not automatically set the
kubectl context to a workload cluster when you create it. You must set the
kubectl context to a workload cluster manually by using the
kubectl config use-context command.
By default, unless you specify the
KUBECONFIG option to save the
kubeconfig for a cluster to a specific file, all workload clusters that you deploy are added to a shared
.kube/config file. If you delete the shared
.kube/config file and you still have the
.kube-tkg/config file for the management cluster, you can recover the
.kube/config of the workload clusters with the
tanzu cluster kubeconfig get CLUSTER-NAME command.
Do not change context or edit the
.kube/config files while Tanzu Kubernetes Grid operations are running.
Proceed to the cluster configuration topic for your infrastructure: