Customizing Clusters, Plans, and Packages with ytt Overlays

This topic explains how you can use ytt Overlays to customize Tanzu Kubernetes (workload) clusters, cluster plans, and Packages.

Tanzu Kubernetes Grid distributes configuration files for clusters and cluster plans in the ~/.config/tanzu/tkg/providers/ directory.

You can customize these configurations by adding or modifying configuration files directly, or by using ytt overlays.

Directly customizing configuration files is simpler, but if you are comfortable with ytt overlays, they let you customize configurations at different scopes and manage multiple, modular configuration files, without destructively editing upstream and inherited configuration values.

ytt Behavior and Conventions

Behaviors and conventions for ytt processing include:

Precedence: ytt traverses directories depth-first in filename alphabetical order, and overwrites duplicate settings as it proceeds. When there are duplicate definitions, the one that ytt processes last takes precedence.

Overlay Types: different ytt overlay types change or set different things:

  • Data values files set or modify configuration values without modifying the structures of objects. These include Bill of Materials (BoM) files and, by convention, files with data in their filenames.

  • Overlay files make changes to object structure definitions. By convention, these files have overlay in their filenames.

For more information about ytt, including overlay examples and an interactive validator tool, see:

Clusters and Cluster Plans

The ~/.config/tanzu/tkg/providers/ directory includes ytt directories and overlay.yaml files at different levels, which lets you scope configuration settings at each level.

  • Provider- and version-specific ytt directories. For example, ~/.config/tanzu/tkg/providers/infrastructure-aws/v0.6.4/ytt.
    • Specific configurations for provider API version.
    • The base-template.yaml file contains all-caps placeholders such as "${CLUSTER_NAME}" and should not be edited.
    • The overlay.yaml file is tailored to overlay values into base-template.yaml.
  • Provider-wide ytt directories. For example, ~/.config/tanzu/tkg/providers/infrastructure-aws/ytt.
    • Provider-wide configurations that apply to all versions.
  • Top-level ytt directory, ~/.config/tanzu/tkg/providers/ytt.
    • Cross-provider configurations.
    • Organized into numbered directories, and processed in number order.
    • You can create a /04_user_customizations subdirectory for configurations that take precedence over any in lower-numbered ytt subdirectories.

Important: You can only use ytt overlays to modify workload clusters. Using ytt overlays to modify management clusters is not supported.

Cluster and Plan Examples

Examples of ytt overlays for customizing workload clusters and cluster plans include:

Applying Overlays to Packages

You can apply or migrate overlays to the user managed packages.

Note: Tanzu package plugin does not natively support annotation. However, Carvel APIs expose an annotation that you can set on PackageInstall CR. Tanzu package plugin preserves this annotation to update package.​

To apply an overlay:

  1. Update the package configuration values to meet your requirements.
  2. Install the package.

    tanzu package install my-pkg –p contour.tanzu.vmware.com -v 1.15.1+vmware.1-tkg.1 --values-file my-values.yaml​

    Here is an example of values.yaml file:

    ```
    infrastructure_provider: vsphere
    namespace: tanzu-system-ingress
    contour:
      configFileContents: {}
      useProxyProtocol: false
      replicas: 2
      pspNames: "vmware-system-restricted"
      logLevel: info
    envoy:
      service:
        type: LoadBalancer
        annotations: {}
        nodePorts:
          http: null
          https: null
        externalTrafficPolicy: Cluster
        aws:
          LBType: classic
        disableWait: false
      hostPorts:
        enable: true
        http: 80
        https: 443
      hostNetwork: false
      terminationGracePeriodSeconds: 300
      logLevel: info
      pspNames: null
    certificates:
      duration: 8760h
      renewBefore: 360h
    ```
    
  3. Create a secret for the overlay. For example:

    kubectl create secret generic my-overlay --from-file=my-overlay.yaml​

  4. Annotate PackageInstall.

    kubectl annotate PackageInstall my-pkg ext.packaging.carvel.dev/ytt-paths-from-secret-name.0=my-overlay

check-circle-line exclamation-circle-line close-line
Scroll to top icon