This topic describes how to migrate Tanzu Kubernetes Grid extensions to packages. You migrate to packages after you upgrade your clusters to v1.4.x. Tanzu Kubernetes Grid packages are deployed and managed by kapp-controller from the Carvel Tools.

Warning: Do not migrate extensions to packages for vSphere 7 clusters. See vSphere 7 Does Not Support Packages.

You should migrate the following extensions to packages on clusters that have been upgraded to Tanzu Kubernetes Grid v1.4.x:

  • Cert Manager
  • Contour
  • Harbor
  • Fluent Bit
  • Prometheus
  • Grafana
  • ExternalDNS

If you are currently using custom ytt overlays for any of your extensions, see Migrate an Extension with Custom Overlays to a Package.

Prerequisites

This procedure assumes that you are upgrading to Tanzu Kubernetes Grid v1.4.x from v1.3.x.

To migrate Tanzu Kubernetes Grid extensions to packages:

Migrate the Cert Manager Extension to a Package

The Cert Manager package is required for all extensions except Fluent Bit. Follow these steps to migrate your Cert Manager extension to a package.

  1. In a terminal, set the context of kubectl to the Tanzu Kubernetes workload cluster where you want to migrate your Cert Manager extension to a package.

  2. Change directory to the extension bundle for the current version of the Cert Manager extension.

    • v1.3.1 tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0 tkg-extensions-v1.3.0+vmware.1/extensions
  3. Retrieve a list of apps installed with the existing extensions bundle:

    kubectl get apps -A
    
  4. From the output of the previous step, record the namespace and app name for the existing Cert Manager extension. For example, cert-manager and cert-manager from the output:

    NAMESPACE              NAME             DESCRIPTION           SINCE-DEPLOY   AGE
    cert-manager           cert-manager     Reconcile succeeded   71s                      23m
    tkg-system             antrea           Reconcile succeeded   25s                      4d5h
    tkg-system             metrics-server   Reconcile succeeded   32s            4d5h
    
    
  5. Confirm that the cert-manager package is available in your workload cluster:

    tanzu package available list -A
    
  6. Retrieve the version of the available package:

    tanzu package available list cert-manager.tanzu.vmware.com -A
    | Retrieving package versions for cert-manager.tanzu.vmware.com...
      NAME                           VERSION                          RELEASED-AT           NAMESPACE
      cert-manager.tanzu.vmware.com  1.1.0+vmware.1-tkg.2             2020-11-24T18:00:00Z  tanzu-package-repo-global
    
  7. Install the Cert Manager package:

    tanzu package install APP-NAME --package-name cert-manager.tanzu.vmware.com --namespace TARGET-NAMESPACE --version AVAILABLE-PACKAGE-VERSION
    

    Where:

    • APP-NAME is the app name you recorded in step 4. For example, cert-manager.
    • TARGET-NAMESPACE is the namespace you recorded in step 4. For example, cert-manager. If this flag is not specified, the Tanzu CLI installs the package in the default namespace.
    • AVAILABLE-PACKAGE-VERSION is the version that you retrieved above.

    For example:

    tanzu package install cert-manager --package-name cert-manager.tanzu.vmware.com --namespace cert-manager --version 1.1.0+vmware.1-tkg.2
    / Installing package 'cert-manager.tanzu.vmware.com'
    | Getting package metadata for 'cert-manager.tanzu.vmware.com'
    | Creating service account 'cert-manager-cert-manager-sa'
    | Creating cluster admin role 'cert-manager-cert-manager-cluster-role'
    | Creating cluster role binding 'cert-manager-cert-manager-cluster-rolebinding'
    \ Creating package resource
    | Package install status: Reconciling
    
    Added installed package 'cert-manager' in namespace 'cert-manager'
    
  8. Confirm that the cert-manager package has been installed:

    tanzu package installed list -A
    

    For example:

    tanzu package installed list -A
    / Retrieving installed packages...
    NAME            PACKAGE-NAME                     PACKAGE-VERSION                  STATUS               NAMESPACE
    cert-manager    cert-manager.tanzu.vmware.com    1.1.0+vmware.1-tkg.2             Reconcile succeeded  my-packages
    antrea          antrea.tanzu.vmware.com                                           Reconcile succeeded  tkg-system
    metrics-server  metrics-server.tanzu.vmware.com                                   Reconcile succeeded  tkg-system
    vsphere-cpi     vsphere-cpi.tanzu.vmware.com                                      Reconcile succeeded  tkg-system
    vsphere-csi     vsphere-csi.tanzu.vmware.com                                      Reconcile succeeded  tkg-system
    

    The cert-manager package and its resources, such as the cert-manager app, are installed in the namespace that you specify when running the tanzu package install command.

  9. Confirm that the cert-manager app has been successfully reconciled in your TARGET-NAMESPACE. For example:

    kubectl get apps -A
    NAMESPACE     NAME             DESCRIPTION           SINCE-DEPLOY   AGE
    cert-manager  cert-manager     Reconcile succeeded   3m2s           3m12s
    ...
    

    If the status is not Reconcile Succeeded, view the full status details of the cert-manager app. Viewing the full status can help you to troubleshoot the problem.

    kubectl get app cert-manager --namespace TARGET-NAMESPACE -o yaml
    

    Where TARGET-NAMESPACE is the namespace in which you installed the package.

  10. Confirm that the cert-manager- pods are running:

    kubectl get pods -A
    
    NAMESPACE      NAME                                                        READY   STATUS    RESTARTS   AGE
    cert-manager   cert-manager-78897c8dc5-pfh7s                               1/1     Running   0          2m21s
    cert-manager   cert-manager-cainjector-86cdb8577c-nrr2s                    1/1     Running   0          2m21s
    cert-manager   cert-manager-webhook-ff45bc699-k8vdd                        1/1     Running   0          2m21s
    ...
    

    The Cert Manager pods and any other resources associated with the Cert Manager component are created in the cert-manager namespace.

Migrate the Contour Extension to a Package

To migrate your Contour extension to a package, follow these steps to record any existing custom configurations, install the Contour package, and copy the configurations over. The Contour package is required if you plan to install the Harbor or Grafana packages. It is also required for ExternalDNS if you plan to configure ExternalDNS to work with Contour resources.

  1. The Contour package depends on the Cert Manager package. Migrate your Cert Manager extension to a package by following the steps in Migrate the Cert Manager Extension to a Package above.

  2. In a terminal, set the context of kubectl to the Tanzu Kubernetes cluster where you want to migrate your Contour extension to a package.

  3. Change directory to the extension bundle for the current version of the Contour extension.

    • v1.3.1: tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0: tkg-extensions-v1.3.0+vmware.1/extensions
  4. Retrieve a list of apps installed with the existing extensions bundle:

    kubectl get apps -A
    
  5. From the output of the previous step, record the namespace and app name for the existing Contour extension. For example, tanzu-system-ingress and contour from the output:

    NAMESPACE              NAME             DESCRIPTION           SINCE-DEPLOY   AGE
    tanzu-system-ingress   contour          Reconcile succeeded   71s            23m
    tkg-system             antrea           Reconcile succeeded   25s            4d5h
    tkg-system             metrics-server   Reconcile succeeded   32s            4d5h
    
  6. Change directories to ingress/contour.

  7. Obtain the current contour-data-values.yaml and secret used by the current Contour extension.

    kubectl get secret contour-data-values -n tanzu-system-ingress -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > contour-v13-data-values.yaml
    
  8. Manually copy any customizations in contour-data-values.yaml to use in the next step. For example, you may have customized NodePort or infrastructure_provider.

  9. If you are upgrading from Tanzu Kubernetes Grid v1.3.1, skip this step. If you are upgrading from v1.3.0, delete the current extension.

    kubectl delete -f ingress/contour/contour-extension.yaml
    
  10. Install the Contour package by following the prerequisites and procedure in Implementing Ingress Control with Contour with the following exceptions:

    • When you run the package install command, use the namespace and app name of the existing Contour extension that you recorded in step 5.
    • Use the customizations from step 8 in the new configuration file for the Contour package. Otherwise, the installation will fail.
  11. Delete the Contour application so that Kubernetes re-creates it.

    kubectl delete app contour -n tanzu-system-ingress
    

Migrate the Harbor Extension to a Package

To migrate your Harbor extension to a package, follow these steps to record any existing custom configurations, install the Harbor package, and copy the configurations over.

  1. The Harbor package depends on the Cert Manager package. Migrate your Cert Manager extension to a package by following the steps in Migrate the Cert Manager Extension to a Package above.

  2. The Harbor package depends on the Contour package. Migrate your Contour extension to a package by following the steps in Migrate the Contour Extension to a Package above.

  3. In a terminal, set the context of kubectl to the Tanzu Kubernetes cluster where you want to migrate your Harbor extension to a package.

  4. Retrieve a list of apps installed with the existing extensions bundle:

    kubectl get apps -A
    
  5. From the output of the previous step, record the namespace and app name for the existing Harbor extension. For example, tanzu-system-registry and harbor from the output:

    NAMESPACE              NAME             DESCRIPTION           SINCE-DEPLOY   AGE
    tanzu-system-registry  harbor           Reconcile succeeded   71s            23m
    tkg-system             antrea           Reconcile succeeded   25s            4d5h
    tkg-system             metrics-server   Reconcile succeeded   32s            4d5h
    
  6. Change directory to the extension bundle for the current version of the extension.

    • v1.3.1: tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0: tkg-extensions-v1.3.0+vmware.1/extensions
  7. Change directories to registry/harbor.

  8. Obtain the harbor-data-values.yaml and secret used by the current Harbor extension.

    kubectl get secret harbor-data-values -n tanzu-system-registry -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > harbor-v13-data-values.yaml
    
  9. Manually copy any customizations in harbor-data-values.yaml to use in the next step. For example, you may need to copy values for namespace, hostname, harborAdminPassword, secretKey, core.secret, persistence, and so on.

  10. If you are upgrading from Tanzu Kubernetes Grid v1.3.1, skip this step. If you are upgrading from v1.3.0, delete the current extension.

    kubectl delete -f registry/harbor/harbor-extension.yaml
    
  11. Install the Harbor package by following the prerequisites and procedure in Deploy Harbor Registry as a Shared Service with the following exceptions:

    • When you run the package install command, use the namespace and app name of the existing Harbor extension that you recorded in step 5.
    • Use the customizations from step 9 in the new configuration file for the Harbor package. Otherwise, the installation will fail.

Migrate the Fluent Bit Extension to a Package

To migrate your Fluent Bit extension to a package, follow these steps to record any custom configurations, uninstall the extension, create the configuration file based on the new values schema, and install the package.

  1. In a terminal, set the context of kubectl to the Tanzu Kubernetes cluster where you want to migrate your Fluent Bit extension to a package.

  2. Change directory to the extension bundle for the current version of the extension.

    • v1.3.1: tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0: tkg-extensions-v1.3.0+vmware.1/extensions
  3. Change directories to logging/fluent-bit.

  4. Obtain the fluent-bit-data-values.yaml files and secrets used by the current Fluent Bit extension.

    Elastic Search

    kubectl get secret fluent-bit-data-values -n tanzu-system-logging -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > elasticsearch/fluent-bit-v13-data-values.yaml
    

    Kafka

    kubectl get secret fluent-bit-data-values -n tanzu-system-logging -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > kafka/fluent-bit-v13-data-values.yaml
    

    Splunk

    kubectl get secret fluent-bit-data-values -n tanzu-system-logging -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > splunk/fluent-bit-v13-data-values.yaml
    

    HTTP

    kubectl get secret fluent-bit-data-values -n tanzu-system-logging -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > http/fluent-bit-v13-data-values.yaml
    

    syslog (Only applicable if you are upgrading from v1.3.0 or later)

    kubectl get secret fluent-bit-data-values -n tanzu-system-logging -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > syslog/fluent-bit-v13-data-values.yaml
    
  5. Obtain the Fluent Bit values schema to use in the next step:

    tanzu package available get fluent-bit.tanzu.vmware.com/1.7.5+vmware.1-tkg.1 --values-schema
    
  6. Create a configuration file for the Fluent Bit package using the values from step 4 and values schema from step 5. For more information about the configuration file, see the Fluent Bit documentation.

  7. If you are upgrading from Tanzu Kubernetes Grid v1.3.1, skip this step. If you are upgrading from v1.3.0, delete the current extension.

    kubectl delete -f logging/fluent-bit/fluent-bit-extension.yaml
    
  8. Use the configuration file you created in step 6 to install the Fluent Bit package by following the prerequisites and procedure in Implementing Log Forwarding with Fluent Bit.

  9. Delete the Fluent Bit application so that Kubernetes re-creates it.

    kubectl delete app fluent-bit -n tanzu-system-logging
    

Migrate the Prometheus Extension to a Package

To migrate your Prometheus extension to a package, follow these steps to record any custom configurations, uninstall the extension, create the configuration file based on the new values schema, and install the package. The Prometheus package is required if you plan to use the Grafana package.

  1. The Prometheus package depends on the Cert Manager package. Migrate your Cert Manager extension to a package by following the steps in Migrate the Cert Manager Extension to a Package above.

  2. The Prometheus package depends on the Contour package if you plan to enable the Contour-based ingress. If so, migrate your Contour extension to a package by following the steps in Migrate the Contour Extension to a Package.

  3. In a terminal, set the context of kubectl to the Tanzu Kubernetes cluster where you want to migrate the Prometheus extension to a package.

  4. Change directory to the extension bundle for the current version of the extension.

    • v1.3.1: tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0: tkg-extensions-v1.3.0+vmware.1/extensions
  5. Change directories to monitoring/prometheus.

  6. Obtain the prometheus-data-values.yaml files and secrets used by the current Prometheus extension.

    kubectl get secret prometheus-data-values -n tanzu-system-monitoring -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > prometheus-v13-data-values.yaml
    
  7. Obtain the Prometheus values schema to use in the next step:

    tanzu package available get prometheus.tanzu.vmware.com/2.27.0+vmware.1-tkg.1 --values-schema
    
  8. Create a configuration file for the Prometheus package using the values from step 6 and values schema from step 7.

  9. If you are upgrading from Tanzu Kubernetes Grid v1.3.1, skip this step. If you are upgrading from v1.3.0, delete the current extension.

    kubectl delete -f monitoring/prometheus/prometheus-extension.yaml
    
  10. Use the configuration file you created in step 8 to install the Prometheus package by following the prerequisites and procedure in Implementing Monitoring with Prometheus and Grafana.

Migrate the Grafana Extension

To migrate your Grafana extension to a package, follow these steps to record any custom configurations, uninstall the extension, create the configuration file based on the new values schema, and install the package.

The Grafana package depends on the Cert Manager, Contour, and Prometheus packages.

  1. Migrate your Cert Manager extension to a package by following the steps in Migrate the Cert Manager Extension to a Package above.

  2. Migrate your Contour extensions to a package by following the steps in Migrate the Contour Extension to a Package above.

  3. Migrate your Prometheus extension to a package by following the steps in Migrate the Prometheus Extension to a Package above.

  4. In a terminal, set the context of kubectl to the Tanzu Kubernetes cluster where you want to migrate your Grafana extension to a package.

  5. Change directory to the extension bundle for the current version of the extension.

    • v1.3.1: tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0: tkg-extensions-v1.3.0+vmware.1/extensions
  6. Change directories to monitoring/grafana.

  7. Obtain the grafana-data-values.yaml files and secrets used by the current Grafana extension.

    kubectl get secret grafana-data-values -n tanzu-system-monitoring -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > grafana-v13-data-values.yaml
    
  8. Obtain the Grafana values schema to use in the next step:

    tanzu package available get grafana.tanzu.vmware.com/7.5.7+vmware.1-tkg.1 --values-schema
    
  9. Create a configuration file for the Grafana package using the values from step 7 and values schema from step 8.

  10. If you are upgrading from Tanzu Kubernetes Grid v1.3.1, skip this step. If you are upgrading from v1.3.0, delete the current extension.

    kubectl delete -f monitoring/grafana/grafana-extension.yaml
    
  11. Use the configuration file you created in step 9 to install the Grafana package by following the prerequisites and procedure in Implementing Monitoring with Prometheus and Grafana.

Migrate the ExternalDNS Extension to a Package

To migrate your ExternalDNS extension to a package, follow these steps to record any existing custom configurations, delete the extension, install the package, and copy the configurations over.

The ExternalDNS package depends on the Cert Manager, Contour, and Harbor Packages.

  1. Migrate your Cert Manager extension to a package by following the steps in Migrate the Cert Manager Extension to a Package above.

  2. Migrate your Contour extension to a package by following the steps in Migrate the Contour Extension to a Package above.

  3. Migrate your Harbor extension to a package by following the steps in Migrate the Harbor Extension to a Package.

  4. In a terminal, set the context of kubectl to the Tanzu Kubernetes cluster where you want to migrate your ExternalDNS extension to a package.

  5. Change directory to the extension bundle for the current version of the extension.

    • v1.3.1: tkg-extensions-v1.3.1+vmware.1/extensions
    • v1.3.0: tkg-extensions-v1.3.0+vmware.1/extensions
  6. Obtain the external-dns-data-values.yaml files and secrets used by the current ExternalDNS extension.

    kubectl get secret external-dns-data-values -n tanzu-system-service-discovery -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > external-dns-v13-data-values.yaml
    
  7. Manually copy any customizations from the current-external-dns-data-values.yaml to use in the configuration file for the ExternalDNS package. For example, you may need to copy values for namespace.

  8. If you are upgrading from Tanzu Kubernetes Grid v1.3.1, skip this step. If you are upgrading from v1.3.0, delete the current extension.

    kubectl delete -f registry/dns/external-dns-extension.yaml
    

    Note Updates to services or Contour resources will not trigger updates to the DNS server after deleting the ExternalDNS extension and before installing the ExternalDNS package. However, the DNS entries created by the ExternalDNS extension should continue to exist during this period.

  9. Install the ExternalDNS package by following the prerequisites and procedure in Implementing Service Discovery with ExternalDNS. If you recorded any custom configurations in step 7, add them to the configuration file for the ExternalDNS package. Otherwise, the installation will fail.

  10. Delete the ExternalDNS application so that Kubernetes re-creates it.

    ckubectl delete app external-dns -n tanzu-system-registry
    

Migrate an Extension with Custom Overlays to a Package

If you previously applied custom ytt overlays to any of the extensions provided by Tanzu Kubernetes Grid, after upgrading your clusters to Tanzu Kubernetes Grid v1.4, you have two options:

To migrate an extension, do the following:

  1. In a terminal, set the context of kubectl to the cluster where you want to migrate your extension to a package.

  2. Save values.yaml from the extension secret to a file:

    kubectl get secret SECRET-NAME -n SECRET-NAMESPACE -o jsonpath={.data.values\\.yaml} | base64 –d > data-values.yaml
    

    You will use this file to create the configuration file for the package.

    For example:

    kubectl get secret contour-data-values -n tanzu-system-ingress -o jsonpath={.data.values\\.yaml} | base64 –d > data-values.yaml
    
  3. Save overlays.yaml from the extension secret to a file:

    kubectl get secret SECRET-NAME -n SECRET-NAMESPACE -o jsonpath={.data.overlays\\.yaml} | base64 -d > overlays.yaml
    

    Because ytt markers are currently not allowed in the configuration file that you provide when running the tanzu package install command, you will use this file after installing the package.

    For example:

    kubectl get secret contour-data-values -n tanzu-system-ingress -o jsonpath={.data.overlays\\.yaml} | base64 -d > overlays.yaml
    
  4. Review the sections of this topic that apply to your extension and follow the steps that they provide to install the package. To prepare the configuration file for the package, use the configuration information that you saved to the data-values.yaml file.

  5. After installing the package, create a secret for your overlays:

    kubectl create secret generic SECRET-NAME --from-file=overlays.yaml
    

    For example:

    kubectl create secret generic custom-annotations --from-file=overlays.yaml
    
  6. Annotate the PackageInstall API resource for the package:

    kubectl annotate PackageInstall INSTALLED-PACKAGE-NAME ext.packaging.carvel.dev/ytt-paths-from-secret-name.NUMBER=SECRET-NAME
    

    Where:

    • INSTALLED-PACKAGE-NAME is the name that you assigned to the package when you installed it.
    • NUMBER must be set to 0 if you are applying custom annotations to the resource for the first time.
    • SECRET-NAME is the name of the secret that you created for your overlays.

    For example:

    kubectl annotate PackageInstall contour ext.packaging.carvel.dev/ytt-paths-from-secret-name.0=custom-annotations
    
check-circle-line exclamation-circle-line close-line
Scroll to top icon