This topic explains how you can use ytt Overlays to customize Tanzu Kubernetes (workload) clusters, cluster plans, extensions, and shared services.

Tanzu Kubernetes Grid distributes configuration files for clusters and cluster plans in the ~/.tanzu/tkg/providers/ directory, and for extensions and shared services in the Tanzu Kubernetes Grid Extensions Bundle.

You can customize these configurations by adding or modifying configuration files directly, or by using ytt overlays.

Directly customizing configuration files is simpler, but if you are comfortable with ytt overlays, they let you customize configurations at different scopes and manage multiple, modular configuration files, without destructively editing upstream and inherited configuration values.

For more information about ytt, including overlay examples and an interactive validator tool, see:

Clusters and Cluster Plans

The ~/.tanzu/tkg/providers/ directory includes ytt directories and overlay.yaml files at different levels, which lets you scope configuration settings at each level:

  • Provider- and version-specific ytt directories. For example, ~/.tanzu/tkg/providers/infrastructure-aws/v0.6.4/ytt.
    • Specific configurations for provider API version.
    • The base-template.yaml file contains all-caps placeholders such as "${CLUSTER_NAME}" and should not be edited.
    • The overlay.yaml file is tailored to overlay values into base-template.yaml.
  • Provider-wide ytt directories. For example, ~/.tanzu/tkg/providers/infrastructure-aws/ytt.
    • Provider-wide configurations that apply to all versions.
  • Top-level ytt directory, ~/.tanzu/tkg/providers/ytt.
    • Cross-provider configurations.
    • Organized into numbered directories, and processed in number order.
    • ytt traverses directories in alphabetical order and overwrites duplicate settings, so you can create a /04_user_customizations subdirectory for configurations that take precedence over any in lower-numbered ytt subdirectories.

IMPORTANT: You can only use ytt overlays to modify workload clusters. Using ytt overlays to modify management clusters is not supported.

Cluster and Plan Examples

Examples of ytt overlays for customizing workload clusters and cluster plans include:

Extensions and Shared Services

The Tanzu Kubernetes Grid Extensions Bundle includes templates for ytt overlays to implement various customizations.

These ytt overlay templates are in overlay subdirectories in the following locations:

Before deploying an extension, you can use these overlay templates or create and apply your own overlays to the extension as follows:

  1. In the extensions bundle directory, under the extension's /overlays directory, modify an overlay template or create a new overlay to contain your custom values:

    • Existing template: Find and modify the template that fits your need. The template filenames indicate their use; for example change-namespace.yaml and update-registry.yaml.
    • New overlay: Create a new ytt overlay file. For example, to add an annotation to the Grafana extension's HTTP Proxy, create a update-grafana-httproxy.yaml overlay with contents:

      #@ load("@ytt:overlay", "overlay")
      #@ load("@ytt:yaml", "yaml")
      #@overlay/match by=overlay.subset({"kind": "HTTPProxy", "metadata": {"name": "grafana-httpproxy"}})
      ---
      metadata:
      #@overlay/match missing_ok=True
      annotations:
        #@overlay/match missing_ok=True
        dns.alpha.kubernetes.io/hostname: grafana.tkg.vclass.local
      
  2. Save the overlay content as a secret in the extension's namespace. For example with update-grafana-httproxy.yaml above, run:

    kubectl create secret generic grafana-httpproxy --from-file=update-grafana-httpproxy.yaml=update-grafana-httpproxy.yaml -n tanzu-system-monitoring
    
  3. In the extension's deployment file, under extensions/ add a reference to the new secret. For example for the grafana-httproxy secret above, add the following to the file /extensions/monitoring/grafana/grafana-extension.yaml under spec.template.ytt.inline.pathsFrom, after the existing grafana-data-values setting:

          - secretRef:
              name: grafana-httpproxy
    

The examples below show some specific use cases for creating and applying custom overlays.

Extension and Shared Service Examples

Examples of applying ytt overlay files for customizing extensions and shared services include:

For more examples, see the TKG Lab repository and its Step by Step setup guide.

check-circle-line exclamation-circle-line close-line
Scroll to top icon