The Tanzu Kubernetes Grid Service declarative API exposes several parameters for configuring Tanzu Kubernetes clusters. Refer to the list and description of all parameters and usage guidelines to provision and customize your clusters.

Annotated YAML for Provisioning a Tanzu Kubernetes Cluster

Note: The TKG v1alpha1 API is deprecated and should not be used to create new TKG clusters. Use the TKG v1alpha2 API instead. See Provisioning Tanzu Kubernetes Clusters Using the TKGS v1alpha2 API.
The annotated YAML lists all available parameters for provisioning a Tanzu Kubernetes cluster with summarized comments for each field.
Note: The annotated YAML is not validated for provisioning a cluster. Refer to the examples for such guidance: Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API. 
apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TanzuKubernetesCluster
#valid config key must consist of alphanumeric characters, '-', '_' or '.'
metadata:
  name: <tanzu kubernetes cluster name>
  namespace: <vsphere namespace where the cluster will be provisioned>
spec:
  distribution:
    version: <tanzu kubernetes release version string: full, point, short>      
  topology:                               
    controlPlane:
      count: <integer either 1 or 3>
      class: <vm class bound to the target vsphere namespace>
      storageClass: <vsphere storage policy bound to the target vsphere namespace>
      volumes: #optional setting for high-churn control plane component
        - name: <user-defined string>
          mountPath: </dir/path>
          capacity:
            storage: <size in GiB> 
    workers:                              
      count: <integer from 0 to 150>
      class: <vm class bound to the target vsphere namespace>
      storageClass: <vsphere storage policy bound to the target vsphere namespace>
      volumes: #optional setting for high-churn worker node component (such as containerd)
        - name: <user-defined string>
          mountPath: </dir/path>
          capacity:
            storage: <size in GiB>              
  settings: #all spec.settings are optional
    storage: #optional storage settings
      classes: [<array of kubernetes storage classes for dynamic pvc provisioning>]
      defaultClass: <default kubernetes storage class>
    network: #optional network settings
      cni: #override default cni set in the tkgservicesonfiguration spec
        name: <antrea or calico>
      pods: #custom pod network
        cidrBlocks: [<array of pod cidr blocks>]
      services: #custom service network
        cidrBlocks: [<array of service cidr blocks>]
      serviceDomain: <custom service domain>
      proxy: #proxy server for outbound connections
        httpProxy: http://<IP:PORT> 
        httpsProxy: http://<IP:PORT>
        noProxy: [<array of CIDRs to not proxy>]
      trust: #trust fields for custom public certs for tls
        additionalTrustedCAs:
          - name: <first-cert-name>
            data: <base64-encoded string of PEM encoded public cert 1>
          - name: <second-cert-name>
            data: <base64-encoded string of PEM encoded public cert 2>

Parameters for Provisioning Tanzu Kubernetes Clusters

The table lists and describes all parameters and acceptable values for provisioning a Tanzu Kubernetes cluster. For examples, see Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API.
Important: A valid key name must consist only of alphanumeric characters, a dash (such as key-name), an underscore (such as KEY_NAME) or a dot (such as key.name). You cannot use the space character in a key name.
Table 1. Parameters for Provisioning Tanzu Kubernetes Clusters
Name Value Description
apiVersion run.tanzu.vmware.com/v1alpha1 Specifies the version of the Tanzu Kubernetes Grid Service API.
kind TanzuKubernetesCluster Specifies the type of Kubernetes resource to create. The only allowed value is TanzuKubernetesCluster (case-sensitive).
metadata Section for cluster metadata Includes cluster metadata, such as name and namespace. This is standard Kubernetes metadata, so you can use generateName instead of name, add labels and annotations, and so on.
name A user-defined string that accepts alphanumeric characters and dashes, for example: my-tkg-cluster-1 Specifies the name of the cluster to create. Current cluster naming constraints:
  • Name length must be 41 characters or less.
  • Name must begin with a letter.
  • Name may contain letters, numbers, and hyphens.
  • Name must end with a letter or a number.
namespace A user-defined string that accepts alphanumeric characters and dashes, for example: my-sns-1 Identifies the name of the Supervisor Namespace where the cluster will be deployed. This is a reference to a Supervisor Namespace that exists on the Supervisor Cluster.
spec Section for technical specifications for the cluster Includes the specification, expressed in declarative fashion, for the end-state of the cluster, including the node toplogy and Kubernetes software distribution.
distribution Section for specifying the Tanzu Kubernetes Release version Indicates the distribution for the cluster: the Tanzu Kubernetes cluster software installed on the control plane and worker nodes, including Kubernetes itself.
version Alphanumeric string with dashes representing the Kubernetes version, for example: v1.20.2+vmware.1-tkg.1 or v1.20.2 or v1.20 Specifies the software version of the Kubernetes distribution to install on cluster nodes using semantic version notation. Can specify the fully qualified version or use version shortcuts, such as "version: v1.20.2", which is resolved to the most recent image matching that patch version, or "version: v1.20", which is resolved to the most recent matching patch version. The resolved version displays as the "fullVersion" on the cluster description after you have created it.
topology Section for cluster node topologies Includes fields that describe the number, purpose, and organization of cluster nodes and the resources allocated to each. Cluster nodes are grouped into pools based on their intended purpose: either control-plane or worker. Each pool is homogeneous, having the same resource allocation and using the same storage.
controlPlane Section for control plane settings Specifies the topology of the cluster control plane, including the number of nodes (count), type of VM (class), and the storage resources allocated for each node (storageClass).
count An integer that is either 1 or 3 Specifies the number of control plane nodes. The control plane must have an odd number of nodes.
class A system-defined element in the form of a string from an enumerated set, for example: guaranteed-small or best-effort-large Specifies the name of the VirtualMachineClass that describes the virtual hardware settings to be used each node in the pool. This controls the hardware available to the node (CPU and memory) as well as the requests and limits on those resources. See Virtual Machine Classes for Tanzu Kubernetes Clusters.
storageClass node-storage (for example) Identifies the storage class to be used for storage of the disks which store the root file systems of the control plane nodes. Run kubectl describe ns on the namespace to view the available storage classes. The available storage classes for the namespace depend on the storage set by the vSphere administrator. Storage classes associated with the Supervisor Namespace are replicated in the cluster. In other words, the storage class must be available on the Supervisor Namespace to be a valid value for this field. See Configuring and Managing vSphere Namespaces.
volumes
Optional storage setting
  • volumes:
    • name: string
    • mountPath: /dir/path
    • capacity
      • storage: GiB size
 Can specify separate disk and storage parameters for etcd on control plane nodes. See the example Cluster with Separate Disks and Storage Parameters.
workers Section for worker node settings Specifies the topology of the cluster worker nodes, including the number of nodes (count), type of VM (class), and the storage resources allocated for each node (storageClass).
count An integer between 0 and 150, for example: 1 or 2 or 7 Specifies the number of worker nodes in the cluster. A cluster with zero worker nodes can be created, allowing for a cluster with only control plane nodes. There is no hard maximum for the number of worker nodes, but a reasonable limit is 150.
Note: A cluster provisioned with 0 worker nodes is not assigned any load balancer services.
class A system-defined element in the form of a string from an enumerated set, for example: guaranteed-small or best-effort-large Specifies the name of the VirtualMachineClass that describes the virtual hardware settings to be used each node in the pool. This controls the hardware available to the node (CPU and memory) as well as the requests and limits on those resources. See Virtual Machine Classes for Tanzu Kubernetes Clusters.
storageClass node-storage (for example) Identifies the storage class to be used for storage of the disks that store the root file systems of the worker nodes. Run kubectl describe ns on the namespace to list available storage classes. The available storage classes for the namespace depend on the storage set by the vSphere administrator. Storage classes associated with the Supervisor Namespace are replicated in the cluster. In other words, the storage class must be available on the Supervisor Namespace to be valid. See Configuring and Managing vSphere Namespaces.
volumes
Optional storage setting
  • volumes:
    • name: string
    • mountPath: /dir/path
    • capacity
      • storage: GiB size
 Can specify separate disk and storage parameters for container images on worker nodes. See the example Cluster with Separate Disks and Storage Parameters.
settings Section for cluster-specific settings; all spec.settings are optional Identifies optional runtime configuration information for the cluster, including node network details and persistent storage for pods.
storage Section for specifying storage Identifies persistent volume (PV) storage entries for container workloads.
classes Array of one or more user-defined strings, for example: ["gold", "silver"] Specifies named persistent volume (PV) storage classes for container workloads. Storage classes associated with the Supervisor Namespace are replicated in the cluster. In other words, the storage class must be available on the Supervisor Namespace to be a valid value. See the example Cluster with Storage Classes and a Default Class for Persistent Volumes.
defaultClass silver (for example) Specifies a named storage class to be annotated as the default in the cluster. If you do not specify it, there is no default. You do not have to specify one or more classes to specify a defaultClass. Some workloads may require a default class, such as Helm. See the example Cluster with Storage Classes and a Default Class for Persistent Volumes.
network Section marker for networking settings Specifies network-related settings for the cluster.
cni Section marker for specifying the CNI Identifies the Container Networking Interface (CNI) plug-in for the cluster. The default is Antrea, which does not need to be specified for new clusters.
name String antrea or calico Specifies the CNI to use. Antrea and Calico are supported. System configuration sets Antrea as the default CNI. The default CNI can be changed. If using the default, this field does not need to be specified.
services Section marker for specifying Kubernetes services subnets Identifies network settings for Kubernetes services. Default is 10.96.0.0/12.
cidrBlocks Array ["198.51.100.0/12"] (for example) Specifies a range of IP addresses to use for Kubernetes services. Default is 10.96.0.0/12. Must not overlap with the settings chosen for the Supervisor Cluster. Although this field is an array, allowing for multiple ranges, currently only a single IP range is allowed. See the networking examples at Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API.
pods Section marker for specifying Kubernetes pods subnets Specifies network settings for pods. Default is 192.168.0.0/16. Minimum block size is /24.
cidrBlocks Array ["192.0.2.0/16"] (for example) Specifies a range of IP addresses to use for Kubernetes pods. Default is 192.168.0.0/16. Must not overlap with the settings chosen for the Supervisor Cluster. Pods subnet size must be equal to or larger than /24. Although this field is an array, allowing for multiple ranges, currently only a single IP range is allowed. See the networking examples at Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API.
serviceDomain "cluster.local" Specifies the service domain for the cluster. Default is cluster.local.
proxy Section that specifies HTTP(s) proxy configuration for the cluster. If implemented all fields are required. Provides fields for specified proxy settings; will be auto-populated if a global proxy is configured and individual cluster proxy is not configured. See the example Cluster with a Proxy Server.
httpProxy http://<user>:<pwd>@<ip>:<port> Specifies a proxy URL to use for creating HTTP connections outside the cluster.
httpsProxy http://<user>:<pwd>@<ip>:<port> Specifies a proxy URL to use for creating HTTPS connections outside the cluster.
noProxy

Array of CIDR blocks to not proxy, for example: [10.246.0.0/16,192.168.144.0/20,192.168.128.0/20].

Get the required values come from the Workload Network on the Supervisor Cluster: Pod CIDRs, Ingress CIDRs, and Egress CIDRs.

Refer to the image below for what values to include in the noProxy array field.

You must not proxy the subnets used by the Workload Network on the Supervisor Cluster for Pods, Ingress, and Egress.

You do not need to include the Services CIDR from the Supervisor Cluster in the noProxy field. Tanzu Kubernetes clusters do not interact with such services.

The endpoints localhost and 127.0.0.1 are automatically not proxied. You do not need to add them to the noProxy field.

The Pod and Service CIDRs for Tanzu Kubernetes clusters are automatically not proxied. You do not need to add them to the noProxy field.

See the example Cluster with a Proxy Server.
trust Section marker for trust parameters. Accepts no data.
additionalTrustedCAs Accepts an array of certificates with name and data for each. Accepts no data.
name String The name of the TLS certificate.
data String The base64-encoded string of a PEM encoded public certificate.

Get the required noProxy values from the Workload Network on the Supervisor Cluster as shown in the image.

The Workload Network window with the Pod CIDRs, Ingress CIDRs, and Egress CIDRs values highlighted.