Upgrade Management Clusters

Caution: VMware recommends not installing or upgrading to Tanzu Kubernetes Grid v1.5.0-v1.5.3, due to a bug in the versions of etcd in the versions of Kubernetes used by Tanzu Kubernetes Grid v1.5.0-v1.5.3. Tanzu Kubernetes Grid v1.5.4 resolves this problem by incorporating a fixed version of etcd. For more information, see Resolved Issues in the TKG v1.5 Release Notes.

To upgrade your Tanzu Kubernetes Grid (TKG) instance, you must first upgrade the management cluster. You cannot upgrade Tanzu Kubernetes clusters until you have upgraded the management cluster that manages them.

Upgrading the management cluster automatically upgrades the auto-managed packages that it runs.

Management clusters and Tanzu Kubernetes clusters use client certificates to authenticate clients. These certificates are valid for one year. To renew them, upgrade your clusters at least once a year.

Note: After you have installed the v1.5 CLI but before a management cluster has been upgraded, all context-specific CLI command groups (tanzu cluster, tanzu kubernetes-release) plus all of the management-cluster plugin commands except for tanzu mc upgrade and tanzu mc create are unavailable and not included in Tanzu CLI --help output.

Prerequisites

  • You performed the steps in Upgrade Tanzu Kubernetes Grid that occur before the step for upgrading management clusters.
  • If you deployed the previous version of Tanzu Kubernetes Grid in an Internet-restricted environment, you have performed the steps in Prepare an Internet-Restricted Environment, including:

    • Recreate and run the gen-publish-images.sh and download-images.sh scripts with the new component image versions.

    • Set the IP address or FQDN of your local registry as an environment variable:

      export TKG_CUSTOM_IMAGE_REPOSITORY="PRIVATE-REGISTRY"
      

      Where PRIVATE-REGISTRY is the IP address or FQDN of your private registry and the name of the project. For example, custom-image-repository.io/yourproject.

      On Windows platforms, use the SET command instead of export.

  • If the management cluster configuration sets CAPBK_BOOTSTRAP_TOKEN_TTL, to extend the bootstrap token’s TTL during cluster initialization, follow the workaround described in Upgrade ignores custom bootstrap token duration in the v1.5 Release Notes.

Procedure

  1. Run the tanzu login command to see an interactive list of management clusters available for upgrade.

    tanzu login
    
  2. Select the management cluster that you want to upgrade. See List Management Clusters and Change Context for more information.

  3. Get the admin credentials of the cluster. The Tanzu CLI alias mc is short for management-cluster.

    tanzu mc kubeconfig get --admin

    
  4. Connect kubectl to the management cluster.

    kubectl config use-context CLUSTER-NAME-admin@CLUSTER-NAME.
    
  5. If your upgrade meets all of the following conditions, you must perform an extra step:

    • You are upgrading from TKG v1.5.1 or v1.5.2 in an internet-restricted environment
    • Your management cluster uses a custom certificate to access the registry
    • On a previous upgrade attempt, the kapp-controller component entered a crashLoopBackoff state

    If the above conditions are true, run kubectl delete apiservice v1alpha1.data.packaging.carvel.dev before running the tanzu mc upgrade command again.

  6. If the management cluster is running on Azure, set the AZURE_CLIENT_SECRET environment variable before upgrading the cluster:

    export AZURE_CLIENT_SECRET=YOUR-AZURE-CLIENT-SECRET
    
  7. Run the tanzu mc upgrade command and enter y to confirm.

    Note: After you run this command, non-admin users cannot log in to the associated workload clusters until the Pinniped pods finish restarting.

    tanzu mc upgrade
    

    If multiple base VM images in your IaaS account have the same version of Kubernetes that you are upgrading to, use the --os-name option to specify the OS you want. See Selecting an OS During Cluster Upgrade for more information.

    For example, on vSphere if you have uploaded both Photon and Ubuntu OVA templates with Kubernetes v1.22.9, specify --os-name ubuntu to upgrade your management cluster to run on an Ubuntu VM.

    tanzu mc upgrade --os-name ubuntu
    

    To skip the confirmation step when you upgrade a cluster, specify the --yes option.

    tanzu mc upgrade --yes
    

    The upgrade process first upgrades the Cluster API providers for vSphere, Amazon EC2, or Azure that are running in the management cluster. Then, it upgrades the version of Kubernetes in all of the control plane and worker nodes of the management cluster.

    If the upgrade times out before it completes, run tanzu mc upgrade again and specify the --timeout option with a value greater than the default of 30 minutes.

    tanzu mc upgrade --timeout 45m0s
    

    Note: After you have installed the v1.5 CLI but before a management cluster has been upgraded, all context-specific CLI command groups (tanzu cluster, tanzu kubernetes-release) plus all of the management-cluster plugin commands except for tanzu mc upgrade and tanzu mc create are unavailable and not included in Tanzu CLI --help output.

  8. When the upgrade finishes, run the tanzu cluster list command with the --include-management-cluster option again to check that the management cluster has been upgraded.

    tanzu cluster list --include-management-cluster
    

    You see that the management cluster is now running the new version of Kubernetes, but that the Tanzu Kubernetes clusters are still running previous versions of Kubernetes.

     NAME                 NAMESPACE   STATUS    CONTROLPLANE  WORKERS  KUBERNETES         ROLES       PLA
     k8s-1-20-14-cluster   default     running   1/1           1/1      v1.20.14+vmware.1   <none>      dev
     k8s-1-21-8-cluster   default     running   1/1           1/1      v1.21.8+vmware.1   <none>      dev
     mgmt-cluster         tkg-system  running   1/1           1/1      v1.22.9+vmware.1   management  dev
    
  9. Regenerate the admin kubeconfig:

    tanzu management-cluster kubeconfig get --admin
    

    The following is sample output of the command:

    Credentials of cluster 'mgmt' have been saved
    You can now access the cluster by running 'kubectl config use-context mgmt-admin@mgmt'
    

What to Do Next

You can now:

check-circle-line exclamation-circle-line close-line
Scroll to top icon