This topic explains how to use a flat configuration file or Kubernetes-style object spec to configure a Tanzu Kubernetes Grid (TKG) workload cluster before deploying it to vSphere 8 with Tanzu. To configure a workload cluster for deployment to vSphere with a standalone management cluster, see vSphere with Standalone Management Cluster Configuration Files.
For general information about how configure workload clusters by using configuration files and object specs, see Configuration Files and Object Specs.
To use vSphere-specific workload cluster features that require some configuration outside of the cluster’s configuration file or object spec, see Clusters on vSphere.
To configure a workload cluster before deploying it to vSphere with Tanzu, you create a Kubernetes-style object spec file if you are configuring a class-based cluster or a cluster configuration file if you are configuring a TKC cluster. When you pass either of these files to the
-f option of
tanzu cluster create, the Tanzu CLI uses the configuration information defined in the file to connect to your vSphere account and create the resources that the cluster will use.
For information about the above cluster types, see Workload Cluster Types.
To configure a workload cluster for deployment to vSphere 8 with Tanzu:
Create or adapt a
Cluster object spec. The vSphere 8 documentation has example
Cluster object specs to work from:
Set VM types, scale, and other basic cluster configurations in the
topology block of the spec file. For information about the
topology block, see Class-Based Cluster Object and Topology Structure and ClusterClass Topology Variables below.
(Optional) To customize attributes that are not settable in the
Cluster object itself, for example, one-time container interface settings in the cluster infrastructure, see Configure One-Time Infrastructure Settings.
The top-level settings configurable in a
Cluster object with type
tanzukubernetescluster are as follows. See ClusterClass Topology Variables for the
variables that you can configure:
spec: clusterNetwork apiServerPort services cidrBlocks pods cidrBlocks serviceDomain controlPlaneEndpoint host port topology class version rolloutAfter controlPlane metadata replicas nodeDrainTimeout nodeDeletionTimeout machineHealthCheck maxUnhealthy nodeStartupTimeout unhealthyConditions workers machineDeployments metadata - class name failureDomain replicas nodeDrainTimeout nodeDeletionTimeout machineHealthCheck maxUnhealthy nodeStartupTimeout unhealthyConditions variables name value variables name value
These fields are set in the
Cluster object type specification in the Tanzu Framework repository: cluster_types.go.
json:setting indicates whether the field is optional. Optional fields have the
*Workersin the type spec, so
workersis indented under
topologyin the object spec.
- class and
variables options are defined in
Cluster object’s cluster class, which is set as the cluster’s
spec.topology.class value. For example, on vSphere with Tanzu, this is a
ClusterClass object named
tanzukubernetescluster, which is different from a
TanzuKubernetesCluster object, as explained in Workload Cluster Types.
extensionCert and many others, as listed in ClusterClass Topology Variables below. These variables configure and override settings in objects that underlie the cluster object, such as
tanzukubernetescluster cluster class, the default
ClusterClass for TKG on vSphere with Tanzu workload clusters, supports the following variables set in
topology.workers.machineDeployments.variables. Variable settings specific to machine deployments, such as node pools, override global settings.
These variables configure and override settings in objects that underlie the cluster object, such as the
proxy settings set in
Machine objects. This enables users to configure a cluster completely within the
Cluster object spec without having to edit lower-level object specs:
The following topics in the vSphere 8 documentation describe reconfiguring a running cluster by changing its
To create a cluster configuration file for a TKC (legacy) workload cluster on vSphere 8, you can copy an existing configuration file for a previous deployment to vSphere with Tanzu and update it. Alternatively, you can create a file from scratch by using an empty template.
To configure a workload cluster deployed by a vSphere with Tanzu Supervisor, you set variables to define the storage classes, VM classes, service domain, namespace, and other required values with which to create your cluster. The following table lists the variables that you can include in the configuration file for a TKC-based cluster. Alternatively, you can set them as local environment variables.
|Variable||Value type or example||Description|
||Sets node counts.|
||CIDR range||The CIDR range to use for pods. The recommended range is
||The CIDR range to use for the Kubernetes services. The recommended range is
||A standard VM class for vSphere with Tanzu, for example
See Using Virtual Machine Classes with TKG Clusters on Supervisor in the vSphere with Tanzu documentation.
|VM class for control plane nodes|
||VM class for worker nodes|
|Variable||Value type or example||Description|
||String||Overridden by the
This name must comply with DNS hostname requirements as outlined in RFC 952 and amended in RFC 1123 and must be 42 characters or less.
||Namespace; defaults to
||The namespace in which to deploy the cluster. To find the namespace of the Supervisor, run
||Container networking interface for hosted workloads, either Antrea or Calico.|
|Deploy a workload cluster with more control plane nodes than the
||Deploy a workload cluster with more worker nodes than the
||Storage classes available for node customization.|
||Default storage class for control plane or workers.|
||Value returned from
||Default storage class for control plane nodes.|
||Default storage class for worker nodes.|
||String||Node pool name, labels, and taints. A
||JSON list of strings, for example,
||JSON list of key-value pair strings, for example,
You can set the variables above by doing either of the following:
Include them in the cluster configuration file passed to the Tanzu CLI
--file option. For example:
From command line, set them as local environment variables by running
export (on Linux and macOS) or
SET (on Windows) on the command line. For example:
If you want to configure unique proxy settings for a workload cluster, you can set
NO_PROXYas environment variables and then use the Tanzu CLI to create the cluster. These variables take precedence over your existing proxy configuration in vSphere with Tanzu.
Proceed to Create Workload Clusters. After you deploy a workload cluster to vSphere you must configure its node DHCP reservations and endpoint DNS as described in Configure Node DHCP Reservations and Endpoint DNS Record (vSphere Only).