Advanced TKC Configuration with ytt

For advanced configuration of TKC-based workload clusters, to set object properties that are not settable by cluster configuration variables, you can customize TKC and cluster plan configuration files in the local ~/.config/tanzu/tkg/providers/infrastructure-tkg-service-vsphere directory of any bootstrap machine.

You can customize these configurations by adding or modifying configuration files directly, or by using ytt overlays.

Directly customizing configuration files is simpler, but if you are comfortable with ytt overlays, they let you customize configurations at different scopes and manage multiple, modular configuration files, without destructively editing upstream and inherited configuration values. ytt overlays only apply to new workload clusters created using the Tanzu CLI.

For more information about how various forms of TKC cluster configuration work and take precedence, see About TKC Cluster Configuration in About Tanzu Kubernetes Grid.

ytt Behavior and Conventions

Behaviors and conventions for ytt processing include:

Precedence: ytt traverses directories depth-first in filename alphabetical order, and overwrites duplicate settings as it proceeds. When there are duplicate definitions, the one that ytt processes last takes precedence.

Overlay Types: different ytt overlay types change or set different things:

• Data values files set or modify configuration values without modifying the structures of objects. These include Bill of Materials (BoM) files and, by convention, files with data in their filenames.

• Overlay files make changes to object structure definitions. By convention, these files have overlay in their filenames.

For more information about ytt, including overlay examples and an interactive validator tool, see:

• Carvel Tools > ytt > Interactive Playground
• The IT Hollow: Using YTT to Customize TKG Deployments

Clusters and Cluster Plans

The ~/.config/tanzu/tkg/providers/ directory includes ytt directories and overlay.yaml files at different levels, which lets you scope configuration settings at each level.

• Provider- and version-specific ytt directories. For example, ~/.config/tanzu/tkg/providers/infrastructure-tkg-service-vsphere/v1.1.0.
  • Specific configurations for provider API version.
  • The base-template.yaml file contains all-caps placeholders such as "${CLUSTER_NAME}" and should not be edited.
  • The overlay.yaml file is tailored to overlay values into base-template.yaml.
• Provider-wide ytt directories. For example, ~/.config/tanzu/tkg/providers/infrastructure-tkg-service-vsphere/ytt.
  • Provider-wide configurations that apply to all versions.
• Top-level ytt directory, ~/.config/tanzu/tkg/providers/infrastructure-tkg-service-vsphere/ytt.
  • Cross-provider configurations.
  • Organized into numbered directories, and processed in number order.
  • You can create a /04_user_customizations subdirectory for configurations that take precedence over any in lower-numbered ytt subdirectories.