Configuration File to Cluster Class Variable Translations

This topic explains how variables in workload cluster configuration files translate to variable settings within a class-based Cluster objects and their ClusterBootstrap and other subordinate objects in Tanzu Kubernetes Grid (TKG). For an example Cluster object and its subordinate objects, see Example Cluster Object and Its Subordinate Objects below.

Overview: Configuration File and Cluster Class Variables

Configuration file variables and class-based cluster variables differ:

Configuration file variables are:
- Formatted as environment variables, all-caps with underscores.
- Set in a cluster configuration file that has a flat structure, or set as environment variables.
- Listed and described in the Configuration File Variable Reference and Configure a Supervisor-Deployed TKC Cluster (Legacy).
- Example setting:
```
TKG_CUSTOM_IMAGE_REPOSITORY: stg-project.vmware.com
```
Class-based cluster variable settings are nested YAML settings within Cluster or ClusterBootstrap object specs:
- Cluster object settings are:
  - Most of the cluster settings available, and can be changed arbitrarily.
  - Defined by the ClusterClass object set as the cluster’s spec.topology.class value.
  - Set in the controlPlane, workers or variables blocks under spec.topology.
  - Example setting:
```
variables:
- name: imageRepository
  value:
    host: stg-project.vmware.com
```
- ClusterBootstrap settings are a few one-time settings for container networking and other low-level infrastructure that cannot be changed in an existing cluster.

When you run the tanzu cluster create command with a cluster configuration file as described in Create a Class-Based Cluster, the Tanzu CLI performs this translation.

Node Configuration

The table below lists variables for configuring the control plane and worker nodes, and the operating system that the node instances run. For information about the variables in the Configuration File Variables column, see Node Configuration in Configuration File Variable Reference.

In the Class-based Cluster Object Structure column, all name / value pair settings are located in the Cluster object definition under spec.topology.variables. The example code in this column applies to Cluster objects on vSphere; object structure on AWS and Azure may differ.

Configuration File Variables	Class-based Cluster Object Structure
OS_NAME: ubuntu OS_VERSION: 20.04 OS_ARCH: amd64	metadata: annotations: osInfo: ubuntu,20.04,amd64
CONTROL_PLANE_MACHINE_COUNT: 3	controlPlane: … replicas: 3
CONTROLPLANE_SIZE: large SIZE: large Values translate to `machine` settings based on infrastructure. `SIZE` applies to both control plane and workers.	- name: controlPlane value: machine: diskGiB: 40 memoryMiB: 16384 numCPUs: 4
CONTROL_PLANE_NODE_SEARCH_DOMAINS: corp.local, example.com	- name: controlPlane value: … network: … searchDomains: - corp.local - example.com
CONTROL_PLANE_NODE_LABELS: ‘key1=value1,key2=value2’	- name: controlPlane value: … nodeLabels: - key: key1 value: value1 - key: key2 value: value2
WORKER_MACHINE_COUNT: 6	workers: machineDeployments: - class: tkg-worker … replicas: 6
WORKER_SIZE: extra-large SIZE: extra-large Values translate to `machine` settings based on infrastructure. `SIZE` applies to both control plane and workers.	- name: worker value: … machine: diskGiB: 80 memoryMiB: 32768 numCPUs: 8
WORKER_NODE_SEARCH_DOMAINS: corp.local, example.com	- name: worker value: … network: searchDomains: - corp.local - example.com
CUSTOM_TDNF_REPOSITORY_CERTIFICATE: “YPdeNjLW[…]”	- name: customTDNFRepository value: certificate: YPdeNjLW[…]
WORKER_ROLLOUT_STRATEGY: RollingUpdate	workers: machineDeployments: - class: tkg-worker failureDomain: “1” metadata: annotations: run.tanzu.vmware.com/resolve-os-image: os-name=ubuntu,os-version=2004 name: md-0 replicas: 1 strategy: type: RollingUpdate

Pod Security Admission (PSA) Controller

The table below lists variables for configuring Pod Security Standards for Pod Security Admission Controller. For information about the variables in the Configuration File Variables column, see Pod Security Standards for Pod Security Admission Controller in Configuration File Variable Reference.

In the Class-based Cluster Object Structure column, all name / value pair settings are located in the Cluster object definition under spec.topology.variables.

Configuration File Variables	Class-based Cluster Object Structure
POD_SECURITY_STANDARD_DEACTIVATED: false POD_SECURITY_STANDARD_AUDIT: privileged POD_SECURITY_STANDARD_WARN: privileged POD_SECURITY_STANDARD_ENFORCE: baseline	- name: podSecurityStandard value: deactivated: false audit: “privileged” enforce: “privileged” warn: “baseline” auditVersion: “v1.26” enforceVersion: “v1.26” warnVersion: “v1.26” exemptions: namespaces: [“kube-system”, “tkg-system”]

Configuration File Variables

Class-based Cluster Object Structure

 POD_SECURITY_STANDARD_DEACTIVATED: false POD_SECURITY_STANDARD_AUDIT: privileged POD_SECURITY_STANDARD_WARN: privileged POD_SECURITY_STANDARD_ENFORCE: baseline

 - name: podSecurityStandard value: deactivated: false audit: “privileged” enforce: “privileged” warn: “baseline” auditVersion: “v1.26” enforceVersion: “v1.26” warnVersion: “v1.26” exemptions: namespaces: [“kube-system”, “tkg-system”]

Cluster Autoscaler

When ENABLE_AUTOSCALER is true, the Tanzu CLI creates a Deployment object for Cluster Autoscaler and adds Cluster Autoscaler annotations to the Cluster object. For information about the variables in the Configuration File Variables column below, see Cluster Autoscaler in Configuration File Variable Reference.

Configuration File Variables Class-based Cluster and Autoscaler Deployment Object Structure

Configuration File Variables	Class-based Cluster and Autoscaler Deployment Object Structure
AUTOSCALER_SCALE_DOWN_DELAY_AFTER_ADD: 10m AUTOSCALER_SCALE_DOWN_DELAY_AFTER_DELETE: 10s AUTOSCALER_SCALE_DOWN_DELAY_AFTER_FAILURE: 3m AUTOSCALER_SCALE_DOWN_UNNEEDED_TIME: 10m AUTOSCALER_MAX_NODE_PROVISION_TIME: 15m AUTOSCALER_MAX_NODES_TOTAL: 0	In Cluster Autoscaler `Deployment` object: spec: containers: - args: … - –scale-down-delay-after-add=10m - –scale-down-delay-after-delete=10s - –scale-down-delay-after-failure=3m - –scale-down-unneeded-time=10m - –max-node-provision-time=15m - –max-nodes-total=0
AUTOSCALER_MAX_SIZE_0: 12 AUTOSCALER_MIN_SIZE_0: 8 AUTOSCALER_MAX_SIZE_1: 10 AUTOSCALER_MIN_SIZE_1: 6 AUTOSCALER_MAX_SIZE_2: 8 AUTOSCALER_MIN_SIZE_2: 4 If `AUTOSCALER_MAX_SIZE_` or `AUTOSCALER_MIN_SIZE_` are not set, the `annotations` settings take the values of `WORKER_MACHINE_COUNT_*`.	In the `Cluster` object, under `spec.topology.workers`: machineDeployments: - class: tkg-worker metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: “12” cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: “8” … name: md-0 - class: tkg-worker metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: “10” cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: “6” … name: md-1 …

 AUTOSCALER_SCALE_DOWN_DELAY_AFTER_ADD: 10m AUTOSCALER_SCALE_DOWN_DELAY_AFTER_DELETE: 10s AUTOSCALER_SCALE_DOWN_DELAY_AFTER_FAILURE: 3m AUTOSCALER_SCALE_DOWN_UNNEEDED_TIME: 10m AUTOSCALER_MAX_NODE_PROVISION_TIME: 15m AUTOSCALER_MAX_NODES_TOTAL: 0

In Cluster Autoscaler Deployment object:

 spec: containers: - args: … - –scale-down-delay-after-add=10m - –scale-down-delay-after-delete=10s - –scale-down-delay-after-failure=3m - –scale-down-unneeded-time=10m - –max-node-provision-time=15m - –max-nodes-total=0

 AUTOSCALER_MAX_SIZE_0: 12 AUTOSCALER_MIN_SIZE_0: 8 AUTOSCALER_MAX_SIZE_1: 10 AUTOSCALER_MIN_SIZE_1: 6 AUTOSCALER_MAX_SIZE_2: 8 AUTOSCALER_MIN_SIZE_2: 4

If AUTOSCALER_MAX_SIZE_ or AUTOSCALER_MIN_SIZE_ are not set, the annotations settings take the values of WORKER_MACHINE_COUNT_*.

In the Cluster object, under spec.topology.workers:

 machineDeployments: - class: tkg-worker metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: “12” cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: “8” … name: md-0 - class: tkg-worker metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: “10” cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: “6” … name: md-1 …

Proxy and Private Image Registry

This section lists variables for using proxies and private image registries, for example in an internet-restricted deployment.

In the Class-based Cluster Object Structure column, all name / value pair settings are located in the Cluster object definition under spec.topology.variables.

Configuration File Variables	Class-based Cluster Object Structure	Notes
TKG_CUSTOM_IMAGE_REPOSITORY: example.com/yourproject TKG_CUSTOM_IMAGE_REPOSITORY_SKIP_TLS_VERIFY: true	- name: imageRepository value: host: example.com/yourproject tlsCertificateValidation: enabled: false	`tldCertificateValidation.enabled` reverses the boolean setting for `TKG_CUSTOM_IMAGE_REPOSITORY_SKIP_TLS_VERIFY` and is not written if its value is `true`.
TKG_HTTP_PROXY_ENABLED: true TKG_HTTP_PROXY: http://proxy.example.com:80 TKG_HTTPS_PROXY: https://proxy.example.com:3128 TKG_NO_PROXY: .noproxy.example.com,noproxy.example.com,192.168.0.0/24	- name: proxy value: httpProxy: http://proxy.example.com:80 httpsProxy: https://proxy.example.com:3128 noProxy: - .noproxy.example.com - noproxy.example.com - 192.168.0.0/24 - […]	Internally, the Tanzu CLI appends to the `noProxy` list values that are not set in a configuration file, as described in Proxy Configuration.
TKG_PROXY_CA_CERT: “LS0tLSBL[…]” TKG_CUSTOM_IMAGE_REPOSITORY_CA_CERTIFICATE: MIIEpgIH[…]"	- name: trust value: - name: proxy data: LS0tLSBL[…] - name: imageRepository data: MIIEpgIH[…]	Values are base64-encoded CA certificates. `TKG_PROXY_CA_CERT` takes precedence over `TKG_CUSTOM_IMAGE_REPOSITORY_CA_CERTIFICATE`, and `trust` `imageRespository` value is not written if `trust` `proxy` value is set.
ADDITIONAL_IMAGE_REGISTRY_1: “example.com/otherregistry-1” ADDITIONAL_IMAGE_REGISTRY_1_SKIP_TLS_VERIFY: false ADDITIONAL_IMAGE_REGISTRY_1_CA_CERTIFICATE: “LS0tLSBL[…]” ADDITIONAL_IMAGE_REGISTRY_2: “example.com/otherregistry-2” ADDITIONAL_IMAGE_REGISTRY_2_SKIP_TLS_VERIFY: true ADDITIONAL_IMAGE_REGISTRY_3: “example.com/otherregistry-3” ADDITIONAL_IMAGE_REGISTRY_3_SKIP_TLS_VERIFY: false ADDITIONAL_IMAGE_REGISTRY_3_CA_CERTIFICATE: “MIIEpgIH[…]”	- name: additionalImageRegistries value: - caCert: LS0tLSBL[…] host: example.com/otherregistry-1 skipTlsVerify: false - host: example.com/otherregistry-2 skipTlsVerify: true - caCert: MIIEpgIH[…] host: example.com/otherregistry-3 skipTlsVerify: false

Common Variables

The following table lists variables that are common to all target platforms.

For information about the variables in the Configuration File Variables column, see Common Variables for All Target Platforms in Configuration File Variable Reference.

In the Class-based Cluster Object Structure column, all name / value pair settings are located in the Cluster object definition under spec.topology.variables.

Configuration File Variables	Class-based Cluster Object Structure
CLUSTER_NAME: my-cluster NAMESPACE: default	metadata: name: my-cluster namespace: default
CLUSTER_PLAN: dev	metadata: annotations: tkg/plan: dev
INFRASTRUCTURE_PROVIDER: vsphere	topology: class: tkg-vsphere-default-v1.0.0
CLUSTER_API_SERVER_PORT: 6443	- name: apiServerPort value: 6443
CNI: antrea	- name: cni value: antrea
ENABLE_AUDIT_LOGGING: true	- name: auditLogging value: enabled: true
CLUSTER_CIDR: 100.96.0.0/11 SERVICE_CIDR: 100.64.0.0/13	spec: clusterNetwork: pods: cidrBlocks: - 100.96.0.0/11 services: cidrBlocks: - 100.64.0.0/13
CONTROLPLANE_CERTIFICATE_ROTATION_ENABLED: true CONTROLPLANE_CERTIFICATE_ROTATION_DAYS_BEFORE: 65	- name: controlPlaneCertificateRotation value: activate: true daysBefore: 65

Antrea CNI

This section lists variables for configuring a cluster’s Antrea container network interface (CNI). The class-based cluster variables reside in the AntreaConfig object that is referenced under the ClusterBootstrap object’s spec.cni.refName property.

Class-based clusters support many Antrea configuration options that do not have corresponding TKG configuration file variables. For all Antrea configuration options that are settable for class-based Cluster objects, see Feature Gates and other topics in the Antrea documentation.

In the AntreaConfig Object Structure column, all settings are located in the AntreaConfig object definition, under spec.antrea.config.

Configuration File Variables	AntreaConfig Object Structure
ANTREA_DISABLE_UDP_TUNNEL_OFFLOAD: false ANTREA_ENABLE_USAGE_REPORTING: false ANTREA_KUBE_APISERVER_OVERRIDE: “https://192.168.77.100:6443” ANTREA_NO_SNAT: false ANTREA_TRAFFIC_ENCAP_MODE: “encap” ANTREA_TRANSPORT_INTERFACE: “eth0” ANTREA_TRANSPORT_INTERFACE_CIDRS: “10.0.0.2/24”	disableUdpTunnelOffload: false enableUsageReporting: false kubeAPIServerOverride: https://192.168.77.100:6443 noSNAT: false trafficEncapMode: encap transportInterface: eth0 transportInterfaceCIDRs: - 10.0.0.2/24
ANTREA_EGRESS: true ANTREA_IPAM: false ANTREA_MULTICAST: false ANTREA_NETWORKPOLICY_STATS: true ANTREA_NODEPORTLOCAL: true ANTREA_SERVICE_EXTERNALIP: false ANTREA_POLICY: true ANTREA_TRACEFLOW: true	featureGates: Egress: true AntreaIPAM: false Multicast: false NetworkPolicyStats: true NodePortLocal: true ServiceExternalIP: false AntreaPolicy: true AntreaTraceflow: true
ANTREA_PROXY: true ANTREA_PROXY_ALL: false ANTREA_PROXY_LOAD_BALANCER_IPS: true ANTREA_PROXY_NODEPORT_ADDRS: “100.70.70.12” ANTREA_PROXY_SKIP_SERVICES: 10.11.1.2,kube-system/kube-dns	antreaProxy: nodePortAddresses: - 100.70.70.12 proxyAll: false proxyLoadBalancerIPs: true skipServices: - 10.11.1.2 - kube-system/kube-dns
ANTREA_FLOWEXPORTER: false ANTREA_FLOWEXPORTER_ACTIVE_TIMEOUT: “60s” ANTREA_FLOWEXPORTER_COLLECTOR_ADDRESS: “flow-aggregator.svc:4739:tls” ANTREA_FLOWEXPORTER_POLL_INTERVAL: “5s” ANTREA_FLOWEXPORTER_IDLE_TIMEOUT: “15s”	flowExporter: activeFlowTimeout: 60s collectorAddress: flow-aggregator.svc:4739:tls idleFlowTimeout: 15s pollInterval: 5s
ANTREA_NODEPORTLOCAL_ENABLED: true ANTREA_NODEPORTLOCAL_PORTRANGE: 61000-62000	nodePortLocal: enabled: true portRange: 61000-62000
ANTREA_EGRESS_EXCEPT_CIDRS: “10.0.0.0/6”	egress: exceptCIDRs: - 10.0.0.0/6
ANTREA_MULTICAST_INTERFACES: “eth0”	multicastInterfaces: - eth0

vSphere

This section lists variables for deploying workload clusters to vSphere. For information about the variables in the Configuration File Variables column below, see vSphere in Configuration File Variable Reference. Settings listed in the Class-based Object Structure column are in the objects of the kind listed.

Configuration File Variables	Object Structure
TKG_IP_FAMILY: ipv4 VSPHERE_CONTROL_PLANE_ENDPOINT: 10.237.177.161 VSPHERE_REGION: my-region VSPHERE_ZONE: my-zone	kind: VSphereCPIConfig spec: vsphereCPI: ipFamily: ipv4 vmNetwork: excludeExternalSubnetCidr: 10.237.177.161/32 excludeInternalSubnetCidr: 10.237.177.161/32 zone: my-zone kind: Cluster metadata: annotations: tkg.tanzu.vmware.com/cluster-controlplane-endpoint: 10.237.177.161 spec: topology: variables: - name: apiServerEndpoint value: 10.237.177.161
VSPHERE_MTU	kind: Cluster spec: topology: variables: - name: controlPlane value: … network: mtu: 1500 - name: worker value: … network: mtu: 1500
VSPHERE_DATACENTER: /dc0 VSPHERE_DATASTORE: /dc0/datastore/sharedVmfs-0 VSPHERE_FOLDER: /dc0/vm/folder0 VSPHERE_NETWORK: /dc0/network/VM Network VSPHERE_RESOURCE_POOL: /dc0/host/cluster0/Resources/rp0 VSPHERE_SERVER: 10.237.179 VSPHERE_STORAGE_POLICY_ID: my-local-sp VSPHERE_TEMPLATE: /dc0/vm/ubuntu-2004-kube-v1.27.5+vmware.2-tkg.1 VSPHERE_TLS_THUMBPRINT: B7:15:(…):1D:2F	kind: ClusterBootstrap spec: additionalPackages: - refName: tkg-storageclass* valuesFrom: inline: VSPHERE_STORAGE_POLICY_ID: my-local-sp kind: Cluster spec: topology: variables: - name: vcenter value: datacenter: /dc0 datastore: /dc0/datastore/sharedVmfs-0 folder: /dc0/vm/folder0 network: /dc0/network/VM Network resourcePool: /dc0/host/cluster0/Resources/rp0 server: 10.237.179.190 storagePolicyID: my-local-sp template: /dc0/vm/ubuntu-2004-kube-v1.27.5+vmware.2-tkg.1 tlsThumbprint: B7:15:(…):1D:2F
VSPHERE_SSH_AUTHORIZED_KEY: ssh-rsa NzaC1yc2EA[…]==	kind: Cluster spec: topology: variables: - name: user value: sshAuthorizedKeys: - ssh-rsa NzaC1yc2EA[…]==
VSPHERE_CONTROL_PLANE_DISK_GIB: “30” VSPHERE_CONTROL_PLANE_MEM_MIB: “2048” VSPHERE_CONTROL_PLANE_NUM_CPUS: “2” VSPHERE_WORKER_DISK_GIB: “50” VSPHERE_WORKER_MEM_MIB: “4096” VSPHERE_WORKER_NUM_CPUS: “4”	kind: Cluster spec: topology: variables: - name: controlPlane value: machine: diskGiB: 40 memoryMiB: 8192 numCPUs: 2 - name: worker value: machine: diskGiB: 40 memoryMiB: 8192 numCPUs: 2
NTP_SERVERS: time.google.com	kind: Cluster spec: topology: variables: - name: ntpServers value: - time.google.com
VSPHERE_AZ_0: rack1 VSPHERE_AZ_1: rack2 VSPHERE_AZ_2: rack3	kind: Cluster spec: topology: workers: machineDeployments: - class: tkg-worker failureDomain: rack1 name: md-0 - class: tkg-worker failureDomain: rack2 name: md-1 - class: tkg-worker failureDomain: rack3 name: md-2
CONTROL_PLANE_NODE_NAMESERVERS: “10.10.10.10,10.10.10.11” WORKER_NODE_NAMESERVERS: “10.10.10.10,10.10.10.11”	- name: controlplane value: network: nameservers: - 10.10.10.10 - 10.10.10.11 - name: worker value: network: nameservers: - 10.10.10.10 - 10.10.10.11

AWS

This section lists variables for deploying workload clusters to AWS. For information about the variables in the Configuration File Variables column below, see AWS in Configuration File Variable Reference.

In the Class-based Cluster Object Structure column, all name / value pair settings are located in the Cluster object definition under spec.topology.variables.

Configuration File Variables	Class-based Cluster Object Structure
AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN AWS_PROFILE	N/A These settings are not stored in the `Cluster` object.
AWS_REGION: us-east-1	- name: region value: us-east-1
AWS_SSH_KEY_NAME: aws-tkg-clusteradmin	- name: sshKeyName value: aws-tkg-clusteradmin
AWS_LOAD_BALANCER_SCHEME_INTERNAL: false	- name: loadBalancerSchemeInternal value: false
BASTION_HOST_ENABLED: true	- name: bastion value: enabled: true
AWS_NODE_AZ: us-east-1a AWS_PRIVATE_NODE_CIDR: 10.0.0.0/24 AWS_PRIVATE_SUBNET_ID: subnet-0a7d376dde53c77ed AWS_PUBLIC_NODE_CIDR: 10.0.1.0/24 AWS_PUBLIC_SUBNET_ID: subnet-0794d50f57e9801b6 AWS_NODE_AZ_1: us-west-2b AWS_PRIVATE_NODE_CIDR_1: 10.0.2.0/24 AWS_PRIVATE_SUBNET_ID_1: subnet-0c338780824d1c59d AWS_PUBLIC_NODE_CIDR_1: 10.0.3.0/24 AWS_PUBLIC_SUBNET_ID_1: subnet-0addabd635d02ba97 AWS_NODE_AZ_2: ap-southeast-3 AWS_PRIVATE_NODE_CIDR_2: 10.0.4.0/24 AWS_PRIVATE_SUBNET_ID_2: subnet-00b9638e419a6187b AWS_PUBLIC_NODE_CIDR_2: 10.0.5.0/24 AWS_PUBLIC_SUBNET_ID_2: subnet-0ed174ef16a2f43aa	- name: network value: subnets: - az: us-east-1a private: cidr: 10.0.0.0/24 id: subnet-0a7d376dde53c77ed public: cidr: 10.0.1.0/24 id: subnet-0794d50f57e9801b6 - az: us-west-2b private: cidr: 10.0.2.0/24 id: subnet-0c338780824d1c59d public: cidr: 10.0.3.0/24 id: subnet-0addabd635d02ba97 - az: ap-southeast-3 private: cidr: 10.0.4.0/24 id: subnet-00b9638e419a6187b public: cidr: 10.0.5.0/24 id: subnet-0ed174ef16a2f43aa
AWS_VPC_CIDR: 10.0.0.0/16	- name: network value: … vpc: cidr: 10.0.0.0/16
AWS_VPC_ID: vpc-0ce8bdfea218	- name: network value: … vpc: existingID: vpc-0ce8bdfea218
AWS_SECURITY_GROUP_BASTION: sg-1 AWS_SECURITY_GROUP_APISERVER_LB: sg-2 AWS_SECURITY_GROUP_LB: sg-3 AWS_SECURITY_GROUP_CONTROLPLANE: sg-4 AWS_SECURITY_GROUP_NODE: sg-5	- name: network value: … securityGroupOverrides: bastion: sg-1 apiServerLB: sg-2 lb: sg-3 controlPlane: sg-4 node: sg-5
AWS_IDENTITY_REF_NAME: my-aws-id AWS_IDENTITY_REF_KIND: AWSClusterRoleIdentity	- name: identityRef value: name: my-aws-id kind: AWSClusterRoleIdentity
NODE_MACHINE_TYPE: m5.large AWS_NODE_OS_DISK_SIZE_GIB: 80 In a multi-AZ deployment, you can also set `NODE_MACHINE_TYPE_1` and `NODE_MACHINE_TYPE_2`	In the `Cluster` object, under `spec.topology.workers`: machineDeployments: - class: tkg-worker name: md-0 value: instanceType: m5.large rootVolume: sizeGiB: 80
CONTROL_PLANE_MACHINE_TYPE: t3.large AWS_CONTROL_PLANE_OS_DISK_SIZE_GIB: 80	- name: controlPlane value: instanceType: t3.large rootVolume: sizeGiB: 80

Microsoft Azure

This section lists variables for deploying workload clusters to Microsoft Azure. For information about the variables in the Configuration File Variables column below, see Microsoft Azure in Configuration File Variable Reference.

In the Class-based Cluster Object Structure column, all name / value pair settings are located in the Cluster object definition under spec.topology.variables.

Configuration File Variables	Class-based Cluster Object Structure
AZURE_CLIENT_ID AZURE_CLIENT_SECRET AZURE_TENANT_ID	N/A These settings are not exposed in the `Cluster` object.
AZURE_LOCATION: eastus2	- name: location value: eastus2
AZURE_RESOURCE_GROUP: my-azure-rg	- name: resourceGroup value: my-azure-rg
AZURE_SUBSCRIPTION_ID: c789uce3-aaaa-bbbb-cccc-a51b6b0gb405	- name: subscriptionID value: c789uce3-aaaa-bbbb-cccc-a51b6b0gb405
AZURE_ENVIRONMENT: AzurePublicCloud	- name: environment value: AzurePublicCloud
AZURE_SSH_PUBLIC_KEY_B64: “c3NoLXJzYSBB […] vdGFsLmlv”	- name: sshPublicKey value: c3NoLXJzYSBB […] vdGFsLmlv
AZURE_FRONTEND_PRIVATE_IP: 10.0.0.100 Set if `AZURE_ENABLE_PRIVATE_CLUSTER` is `true`.	- name: frontendPrivateIP value: 10.0.0.100
AZURE_CUSTOM_TAGS: “foo=bar, plan=prod”	- name: customTags value: “foo=bar, plan=prod”
AZURE_ENABLE_ACCELERATED_NETWORKING: true	- name: acceleratedNetworking value: enabled: true
AZURE_ENABLE_PRIVATE_CLUSTER: false	- name: privateCluster value: enabled: false
AZURE_VNET_CIDR: 10.0.0.0/16 AZURE_VNET_NAME: my-azure-vnet AZURE_VNET_RESOURCE_GROUP: my-azure-vnet-rg	- name: network value: vnet: cidrBlocks: - 10.0.0.0/16 name: my-azure-vnet resourceGroup: my-azure-vnet-rg
AZURE_IDENTITY_NAME: my-azure-id AZURE_IDENTITY_NAMESPACE: default	- name: identityRef value: name: my-azure-id namespace: default
AZURE_CONTROL_PLANE_DATA_DISK_SIZE_GIB: 256 AZURE_CONTROL_PLANE_OS_DISK_SIZE_GIB: 128 AZURE_CONTROL_PLANE_OS_DISK_STORAGE_ACCOUNT_TYPE: Premium_LRS AZURE_ENABLE_CONTROL_PLANE_OUTBOUND_LB: true AZURE_CONTROL_PLANE_OUTBOUND_LB_FRONTEND_IP_COUNT: 1 AZURE_CONTROL_PLANE_SUBNET_CIDR: 10.0.0.0/24 AZURE_CONTROL_PLANE_SUBNET_NAME: my-azure-cp-subnet AZURE_CONTROL_PLANE_MACHINE_TYPE: Standard_D2s_v3	- name: controlPlane value: dataDisks: - sizeGiB: 256 osDisk: sizeGiB: 128 storageAccountType: Premium_LRS outboundLB: enabled: true frontendIPCount: 1 subnet: cidr: 10.0.0.0/24 name: my-azure-cp-subnet vmSize: Standard_D2s_v3
AZURE_ENABLE_NODE_DATA_DISK: true AZURE_NODE_DATA_DISK_SIZE_GIB: 256 AZURE_NODE_OS_DISK_SIZE_GIB: 128 AZURE_NODE_OS_DISK_STORAGE_ACCOUNT_TYPE: Premium_LRS AZURE_ENABLE_NODE_OUTBOUND_LB: true AZURE_NODE_OUTBOUND_LB_FRONTEND_IP_COUNT: 1 AZURE_NODE_OUTBOUND_LB_IDLE_TIMEOUT_IN_MINUTES: 4 AZURE_NODE_SUBNET_CIDR: 10.0.1.0/24 AZURE_NODE_SUBNET_NAME: my-azure-worker-subnet AZURE_NODE_MACHINE_TYPE: Standard_D2s_v3	- name: worker value: dataDisks: - sizeGiB: 256 osDisk: sizeGiB: 128 storageAccountType: Premium_LRS outboundLB: enabled: true frontendIPCount: 1 idleTimeoutInMinutes: 4 subnet: cidr: 10.0.1.0/24 name: my-azure-worker-subnet vmSize: Standard_D2s_v3

NSX Advanced Load Balancer

This section lists variables for configuring NSX Advanced Load Balancer (ALB) in TKG.

Configuration File Variables	Class-based Cluster Object Structure
AVI_CONTROL_PLANE_HA_PROVIDER	topology: variables: - name: aviAPIServerHAProvider value: true

Example `Cluster` Object and Its Subordinate Objects

When you pass a cluster configuration file to the --file flag of tanzu cluster create, the command converts your cluster configuration file into a cluster spec file. See below for an example Cluster object and its subordinate objects generated by tanzu cluster create from the following configuration file:

CLUSTER_NAME: example-cluster
CLUSTER_PLAN: dev
NAMESPACE: default
CNI: antrea
VSPHERE_NETWORK: /dc0/network/VM Network
VSPHERE_SSH_AUTHORIZED_KEY: ssh-rsa AAAACCCza2EBBB[...]ADGQAg/POl6vyWOmQ==
VSPHERE_USERNAME: [email protected]
VSPHERE_PASSWORD: 1234567AbC!
VSPHERE_SERVER: 10.XXX.XXX.71
VSPHERE_DATACENTER: /dc0
VSPHERE_RESOURCE_POOL: /dc0/host/cluster0/Resources/example-tkg
VSPHERE_DATASTORE: /dc0/datastore/vsanDatastore
VSPHERE_FOLDER: /dc0/vm/example-tkg
VSPHERE_TLS_THUMBPRINT: ""
VSPHERE_INSECURE: true
VSPHERE_CONTROL_PLANE_ENDPOINT: 10.XXX.XXX.75
AVI_CONTROL_PLANE_HA_PROVIDER: false
ENABLE_AUDIT_LOGGING: false
ENABLE_DEFAULT_STORAGE_CLASS: true
CLUSTER_CIDR: 100.96.0.0/11
SERVICE_CIDR: 100.64.0.0/13
ENABLE_AUTOSCALER: false

Resulting Cluster object and its subordinate objects:

apiVersion: cpi.tanzu.vmware.com/v1alpha1
kind: VSphereCPIConfig
metadata:
  name: example-cluster
  namespace: default
spec:
  vsphereCPI:
    mode: vsphereCPI
    tlsCipherSuites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    vmNetwork:
      excludeExternalSubnetCidr: 10.XXX.XXX.75/32
      excludeInternalSubnetCidr: 10.XXX.XXX.75/32
---
apiVersion: csi.tanzu.vmware.com/v1alpha1
kind: VSphereCSIConfig
metadata:
  name: example-cluster
  namespace: default
spec:
  vsphereCSI:
    config:
      datacenter: /dc0
      httpProxy: ""
      httpsProxy: ""
      insecureFlag: true
      noProxy: ""
      region: null
      tlsThumbprint: ""
      useTopologyCategories: false
      zone: null
    mode: vsphereCSI
---
apiVersion: run.tanzu.vmware.com/v1alpha3
kind: ClusterBootstrap
metadata:
  annotations:
    tkg.tanzu.vmware.com/add-missing-fields-from-tkr: TKR-NAME
  name: example-cluster
  namespace: default
spec:
  additionalPackages:
  - refName: metrics-server*
  - refName: secretgen-controller*
  - refName: pinniped*
  cpi:
    refName: vsphere-cpi*
    valuesFrom:
      providerRef:
        apiGroup: cpi.tanzu.vmware.com
        kind: VSphereCPIConfig
        name: example-cluster
  csi:
    refName: vsphere-csi*
    valuesFrom:
      providerRef:
        apiGroup: csi.tanzu.vmware.com
        kind: VSphereCSIConfig
        name: example-cluster
  kapp:
    refName: kapp-controller*
---
apiVersion: v1
kind: Secret
metadata:
  name: example-cluster
  namespace: default
stringData:
  password: 1234567AbC!
  username: [email protected]
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  annotations:
    osInfo: ubuntu,20.04,amd64
    tkg.tanzu.vmware.com/cluster-controlplane-endpoint: 10.XXX.XXX.75
    tkg/plan: dev
  labels:
    tkg.tanzu.vmware.com/cluster-name: example-cluster
  name: example-cluster
  namespace: default
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 100.96.0.0/11
    services:
      cidrBlocks:
      - 100.64.0.0/13
  topology:
    class: tkg-vsphere-default-CLUSTER-CLASS-VERSION
    controlPlane:
      metadata:
        annotations:
          run.tanzu.vmware.com/resolve-os-image: image-type=ova,os-name=ubuntu
      replicas: 1
    variables:
    - name: cni
      value: antrea
    - name: controlPlaneCertificateRotation
      value:
        activate: true
        daysBefore: 90
    - name: auditLogging
      value:
        enabled: false
    - name: podSecurityStandard
      value:
        audit: restricted
        deactivated: false
        warn: restricted
    - name: apiServerEndpoint
      value: 10.XXX.XXX.75
    - name: aviAPIServerHAProvider
      value: false
    - name: vcenter
      value:
        cloneMode: fullClone
        datacenter: /dc0
        datastore: /dc0/datastore/vsanDatastore
        folder: /dc0/vm/example-tkg
        network: /dc0/network/VM Network
        resourcePool: /dc0/host/cluster0/Resources/example-tkg
        server: 10.XXX.XXX.71
        storagePolicyID: ""
        tlsThumbprint: ""
    - name: user
      value:
        sshAuthorizedKeys:
        - ssh-rsa AAAACCCza2EBBB[...]ADGQAg/POl6vyWOmQ==
    - name: controlPlane
      value:
        machine:
          diskGiB: 40
          memoryMiB: 8192
          numCPUs: 2
    - name: worker
      value:
        machine:
          diskGiB: 40
          memoryMiB: 4096
          numCPUs: 2
    - name: security
      value:
        fileIntegrityMonitoring:
          enabled: false
        imagePolicy:
          pullAlways: false
          webhook:
            enabled: false
            spec:
              allowTTL: 50
              defaultAllow: true
              denyTTL: 60
              retryBackoff: 500
        kubeletOptions:
          eventQPS: 50
          streamConnectionIdleTimeout: 4h0m0s
        systemCryptoPolicy: default
    version: KUBERNETES-VERSION
    workers:
      machineDeployments:
      - class: tkg-worker
        metadata:
          annotations:
            run.tanzu.vmware.com/resolve-os-image: image-type=ova,os-name=ubuntu
        name: md-0
        replicas: 1
        strategy:
          type: RollingUpdate

The example object spec file above includes the following placeholder text:

TKR-NAME in tkg.tanzu.vmware.com/add-missing-fields-from-tkr: TKR-NAME: Set by tanzu cluster create to a compatible Tanzu Kubernetes release (TKr), depending on your configuration. For example, tkg.tanzu.vmware.com/add-missing-fields-from-tkr: v1.27.5---vmware.1-tkg.1. v1.27.5---vmware.1-tkg.1 is the default TKr for this Tanzu Kubernetes Grid release.
KUBERNETES-VERSION in version: KUBERNETES-VERSION: Set by tanzu cluster create to a compatible Kubernetes version, depending on your configuration. For example, version: v1.27.5+vmware.1-tkg.1. v1.27.5+vmware.1-tkg.1 is the default Kubernetes version for this Tanzu Kubernetes Grid release.
CLUSTER-CLASS-VERSION in class: tkg-vsphere-default-CLUSTER-CLASS-VERSION: Is the version of the default class. For example, class: tkg-vsphere-default-v1.1.1.

Configuration File to Cluster Class Variable Translations

Overview: Configuration File and Cluster Class Variables

Node Configuration

Pod Security Admission (PSA) Controller

Cluster Autoscaler

Proxy and Private Image Registry

Common Variables

Antrea CNI

vSphere

AWS

Microsoft Azure

NSX Advanced Load Balancer

Example Cluster Object and Its Subordinate Objects

Example `Cluster` Object and Its Subordinate Objects