Tanzu Kubernetes Grid includes binaries for tools that provide in-cluster and shared services to the clusters running in your Tanzu Kubernetes Grid instance. All of the provided binaries and container images are built and signed by VMware.

You can add functionalities to Tanzu Kubernetes clusters by installing extensions to different cluster locations as follows:

Function Extension Location Procedure
Ingress Control Contour Tanzu Kubernetes cluster Implementing Ingress Control with Contour
Log Forwarding Fluent Bit Tanzu Kubernetes cluster Implementing Log Forwarding with Fluentbit
Container Registry Harbor Shared Services cluster Deploy Harbor Registry as a Shared Service
Monitoring Prometheus Tanzu Kubernetes cluster Implementing Monitoring with Prometheus and Grafana
Grafana Tanzu Kubernetes cluster
User Authentication Dex Management cluster Implementing User Authenticaton with Dex and Gangway
Gangway Tanzu Kubernetes cluster

Before you can deploy the Tanzu Kubernetes Grid extensions, you must prepare your bootstrap environment by performing the following tasks.

  1. Download and Unpack the Tanzu Kubernetes Grid Extensions Bundle
  2. Install the Carvel Tools on the Bootstrap Environment
  3. If you are using Tanzu Kubernetes Grid in an Internet-restricted environment, see Deploying the Tanzu Kubernetes Grid Extensions in an Internet Restricted Environment.

Download and Unpack the Tanzu Kubernetes Grid Extensions Bundle

The Tanzu Kubernetes Grid extension manifests are provided in a separate bundle to the Tanzu Kubernetes Grid CLI and other binaries.

  1. On the system that you use as the bootstrap machine, go to https://www.vmware.com/go/get-tkg and log in with your My VMware credentials.
  2. Under Product Downloads, click Go to Downloads.
  3. Scroll to VMware Tanzu Kubernetes Grid Extensions Manifest 1.2.0 and click Download Now.
  4. Use either the tar command or the extraction tool of your choice to unpack the bundle of YAML manifest files for the Tanzu Kubernetes Grid extensions.

    tar -xzf tkg-extensions-manifests-v1.2.0-vmware.1.tar.gz

    For convenience, unpack the bundle in the same location as the one from which you run tkg and kubectl commands.

IMPORTANT: After you unpack the bundle, the extensions files are contained in a folder named tkg-extensions-v1.2.0+vmware.1. This folder contains subfolders for each type of extension, for example, authentication, ingress, registry, and so on. At the top level of the folder there is an additional subfolder named extensions. The extensions folder also contains subfolders for authentication, ingress, registry, and so on. In the procedures to deploy the extensions, take care to run commands from the location provided in the instructions. Commands are usually run from within the extensions folder.

Install the Carvel Tools on the Bootstrap Environment

The Tanzu Kubernetes Grid uses the following tools from the Carvel open-source project to configure and deploy all of the extensions:

  • ytt: YAML templating tool
  • kapp: Kubernetes applications CLI
  • kbld: Kubernetes builder

Tanzu Kubernetes Grid provides signed binaries for ytt, kapp, and kbld that are bundled with the Tanzu Kubernetes Grid CLI. For information about where to download the CLI bundle, see Install the Tanzu Kubernetes Grid CLI.

To deploy the extensions, you update configuration files with information about your environment. You then use kubectl to apply preconfigured YAML files that pull data from the updated configuration files to create and update clusters that implement the extensions. The YAML files include calls to ytt, kapp, and kbld commands, so these tools must be present on your bootstrap environment when you deploy the extensions.

  1. Navigate to the location on your bootstrap environment machine where you saved the Tanzu Kubernetes Grid CLI bundle.

    The bundles are tkg-darwin-amd64-VERSION for macOS, tkg-linux-amd64-VERSION for Linux, and tkg-windows-amd64-VERSION for Windows.

  2. Rename the YTT binary for your platform to ytt and make it available to the system:

    • For macOS and Linux platforms:

      1. Move the binary to /usr/local/bin:
        mv ./ytt-linux-amd64-v0.30.0+vmware.1 /usr/local/bin/ytt
        mv ./ytt-darwin-amd64-v0.30.0+vmware.1 /usr/local/bin/ytt
      2. Confirm that the binary is executable by running the ls command.
    • For Windows platforms:

      1. Create a new Program Files\ytt folder and copy the ytt-windows-amd64-v0.30.0+vmware.1 binary into it.
      2. Rename ytt-windows-amd64-v0.30.0+vmware.1 to ytt.exe.
      3. Right-click the ytt folder, select Properties > Security, and make sure that your user account has the Full Control permission.
      4. Use Windows Search to search for env.
      5. Select Edit the system environment variables and click the Environment Variables button.
      6. Select the Path row under System variables, and click Edit.
      7. Click New to add a new row and enter the path to the ytt binary.
  3. At the command line in a new terminal, run ytt version to check that the correct version of the binary is properly installed.

    You should see information about the installed ytt version.

    ytt version 0.30.0
    
  4. Rename the Kapp binary for your platform to kapp and make it available to the system:

    • For macOS and Linux platforms:

      1. Move the binary to /usr/local/bin:
        mv ./kapp-linux-amd64-v0.33.0+vmware.1 /usr/local/bin/kapp
        mv ./kapp-darwin-amd64-v0.33.0+vmware.1 /usr/local/bin/kapp
      2. Confirm that the binary is executable by running the ls command.
    • For Windows platforms:

      1. Create a new Program Files\kapp folder and copy the kapp-windows-amd64-v0.33.0+vmware.1 binary into it.
      2. Rename kapp-windows-amd64-v0.33.0+vmware.1 to kapp.exe.
      3. Right-click the kapp folder, select Properties > Security, and make sure that your user account has the Full Control permission.
      4. Use Windows Search to search for env.
      5. Select Edit the system environment variables and click the Environment Variables button.
      6. Select the Path row under System variables, and click Edit.
      7. Click New to add a new row and enter the path to the kapp binary.
  5. At the command line in a new terminal, run kapp version to check that the correct version of the binary is properly installed.

    You should see information about the installed kapp version.

    kapp version 0.33.0
    
  6. Rename the Kbld binary for your platform to kbld and make it available to the system:

    • For macOS and Linux platforms:

      1. Move the binary to /usr/local/bin:
        mv ./kbld-linux-amd64-v0.24.0+vmware.1 /usr/local/bin/kbld
        mv ./kbld-darwin-amd64-v0.24.0+vmware.1 /usr/local/bin/kbld
      2. Confirm that the binary is executable by running the ls command.
    • For Windows platforms:

      1. Create a new Program Files\kbld folder and copy the kbld-windows-amd64-v0.24.0+vmware.1 binary into it.
      2. Rename kbld-windows-amd64-v0.24.0+vmware.1 to kbld.exe.
      3. Right-click the kbld folder, select Properties > Security, and make sure that your user account has the Full Control permission.
      4. Use Windows Search to search for env.
      5. Select Edit the system environment variables and click the Environment Variables button.
      6. Select the Path row under System variables, and click Edit.
      7. Click New to add a new row and enter the path to the kbld binary.
  7. At the command line in a new terminal, run kbld version to check that the correct version of the binary is properly installed.

    You should see information about the installed kbld version.

    kbld version 0.24.0
    

Installing Extension Prerequisite Components to a Cluster

Procedures for installing Tanzu Kubernetes Grid extensions on a cluster, whether management or workload, include installing these components:

  • Tanzu Mission Control extension-manager
  • Kapp controller
  • For most extensions: cert-manager

To install these components on a cluster, specify the cluster with kubectl config use-context and then do the following:

  1. Install the Tanzu Mission Control extension manager on the cluster.

    The Tanzu Kubernetes Grid extensions and Tanzu Mission Control both use the same extension-manager service. You must install the extension manager even if you do not intend to use Tanzu Mission Control.

    kubectl apply -f tmc-extension-manager.yaml
    

    You should see confirmation that a namespace, resource definitions, a service account, an RBAC role, and a role binding for the extension-manager service are all created.

    namespace/vmware-system-tmc created
    customresourcedefinition.apiextensions.k8s.io/agents.clusters.tmc.cloud.vmware.com created
    customresourcedefinition.apiextensions.k8s.io/extensions.clusters.tmc.cloud.vmware.com created
    customresourcedefinition.apiextensions.k8s.io/extensionresourceowners.clusters.tmc.cloud.vmware.com created
    customresourcedefinition.apiextensions.k8s.io/extensionintegrations.clusters.tmc.cloud.vmware.com created
    customresourcedefinition.apiextensions.k8s.io/extensionconfigs.intents.tmc.cloud.vmware.com created
    serviceaccount/extension-manager created
    clusterrole.rbac.authorization.k8s.io/extension-manager-role created
    clusterrolebinding.rbac.authorization.k8s.io/extension-manager-rolebinding created
    service/extension-manager-service created
    deployment.apps/extension-manager created
    

    Note: You can disregard any warning messages about deprecated APIs.

  2. Install the Kapp controller on the cluster.

    kubectl apply -f kapp-controller.yaml
    

    You should see confirmation that a service account, resource definition, and RBAC role are created for the kapp-controller service.

    serviceaccount/kapp-controller-sa created
    customresourcedefinition.apiextensions.k8s.io/apps.kappctrl.k14s.io created
    deployment.apps/kapp-controller created
    clusterrole.rbac.authorization.k8s.io/kapp-controller-cluster-role created
    clusterrolebinding.rbac.authorization.k8s.io/kapp-controller-cluster-role-binding created
    
  3. Determine whether the extension image you are installing is stored in a private container registry that uses a self-signed certificate:

    1. Look in the extension manifest file in the extensions directory. The file has a name of the form NAME-extension.yaml, for example harbor-extension.yaml or contour-extension.yaml.

    2. In the manifest, check whether the spec.objects.spec.fetch.image.url field points to your own private registry or a public registry. Private registries typically use self-signed certificates.

  4. If the extension manifest pulls the image from a registry with a self-signed certificate, configure the Kapp controller to trust the registry as follows:

    1. Create a file custom-ca-certs.crt containing the CA certificate of your private container registry.

    2. Create a ConfigMap with the certificate by running:

      kubectl create configmap custom-ca-certs --from-file=custom-ca-certs.crt -n vmware-system-tmc
      
    3. Back up your kapp-controller.yaml and replace the kind: Deployment block with the code below:

      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: kapp-controller
        namespace: vmware-system-tmc
      spec:
        replicas: 1
        revisionHistoryLimit: 0
        selector:
          matchLabels:
            app: kapp-controller
        template:
          metadata:
            labels:
              app: kapp-controller
          spec:
            containers:
            - args: []
              command:
              - /kapp-controller
              env:
              - name: KAPPCTRL_MEM_TMP_DIR
                value: /etc/kappctrl-mem-tmp
              image: registry.tkg.vmware.run/kapp-controller:v0.9.0_vmware.1
              imagePullPolicy: IfNotPresent
              name: kapp-controller
              resources:
                requests:
                  cpu: 120m
                  memory: 100Mi
              securityContext:
                runAsGroup: 2000
                runAsUser: 1000
              volumeMounts:
              - mountPath: /etc/kappctrl-mem-tmp
                name: template-fs
              - mountPath: /etc/ssl/certs/custom-ca-certs.crt
                name: custom-ca-certs
                readOnly: true
                subPath: custom-ca-certs.crt
            securityContext:
              fsGroup: 3000
            serviceAccount: kapp-controller-sa
            volumes:
            - emptyDir:
                medium: Memory
              name: template-fs
            - configMap: 
                name: custom-ca-certs
              name: custom-ca-certs
      
    4. Apply the Kapp controller definition to mount the CA cert file from the ConfigMap:

      kubectl apply -f kapp-controller.yaml
      
  5. For extensions other than Dex or Fluent Bit, deploy cert-manager, which provides automated certificate management, on the cluster.

    kubectl apply -f cert-manager/
    

    Dex and Fluent Bit do not use cert-manager.

Upgrading the Tanzu Kubernetes Grid Extensions

For information about how to upgrade the Tanzu Kubernetes Grid extensions from a previous release, see Upgrade Tanzu Kubernetes Grid Extensions from 1.1.x to 1.2.x.

check-circle-line exclamation-circle-line close-line
Scroll to top icon