Upgrade Workload Clusters

This topic explains how to upgrade Tanzu Kubernetes Grid (TKG) workload clusters. For TKG with a standalone management cluster, you first need to upgrade the management cluster that manages the workload clusters.

Important

  • If you are upgrading workload clusters that you deployed with Supervisor in vSphere 8, see Upgrading Clusters Deployed by Supervisor (vSphere 8 Only) in Creating and Managing TKG 2.2 Workload Clusters with the Tanzu CLI.
  • You cannot upgrade clusters with Windows worker nodes as described in Deploy a Multi-OS Workload Cluster.
  • Standalone management clusters and workload clusters use client certificates to authenticate clients. These certificates are valid for one year. To renew them, upgrade your clusters at least once a year.
  • TKG does not support upgrading clusters running previous versions of tiny TKr. To update a minimal single-node cluster to the latest tiny TKr version, you need to delete the old cluster and create a new one. See Single-Node Clusters on vSphere for information about tiny TKrs.

Prerequisites

Infrastructure Prerequisites

Important

Tanzu Kubernetes Grid v2.4.x is the last version of TKG that supports upgrading existing TKG workload clusters on AWS and Azure. The ability to upgrade TKG workload clusters on AWS and Azure will be removed in the Tanzu Kubernetes Grid v2.5 release.

Starting from now, VMware recommends that you use Tanzu Mission Control to create native AWS EKS and Azure AKS clusters. However, upgrading existing TKG workload clusters on AWS and Azure remains fully supported for all TKG releases up to and including TKG v2.4.x.

For more information, see Deprecation of TKG Management and Workload Clusters on AWS and Azure in the VMware Tanzu Kubernetes Grid v2.4 Release Notes.

vSphere
If you are upgrading clusters that run on vSphere, before you can upgrade clusters to a non-default version of Kubernetes for your version of Tanzu Kubernetes Grid, the appropriate base image template OVAs must be available in vSphere as VM templates. For information about importing OVA files into vSphere, see the vSphere tab in Prepare to Upgrade Clusters.
AWS
If you are upgrading clusters that run on Amazon Web Services (AWS), the Amazon Linux 2 Amazon Machine Images (AMI) that include the supported Kubernetes versions are publicly available to all AWS users, in all supported AWS regions. Tanzu Kubernetes Grid automatically uses the appropriate AMI for the Kubernetes version that you specify during upgrade.
Azure
If you are upgrading clusters that run on Azure, ensure that you completed the steps in the Azure tab in Prepare to Upgrade Clusters.


Procedure

The upgrade process upgrades the version of Kubernetes in all of the control plane and worker nodes of your workload clusters.

  1. To see an interactive list of available management clusters and select the management cluster that manages the clusters that you want to upgrade, run the tanzu login command:

    tanzu login
    
  2. To list your workload clusters, run:

    tanzu cluster list --include-management-cluster -A
    

    The tanzu cluster list command with the --include-management-cluster -A options shows the version of Kubernetes that is running in the management cluster and all of the clusters that it manages. In this example, you can see that the management cluster has already been upgraded to v1.25.7, but the workload clusters are running older versions of Kubernetes.

    NAME                 NAMESPACE   STATUS    CONTROLPLANE  WORKERS  KUBERNETES         ROLES       PLAN  TKR
    k8s-1-23-16-cluster  default     running   1/1           1/1      v1.23.16+vmware.1  <none>      dev   v1.23.16---vmware.1-tkg.1
    k8s-1-24-10-cluster  default     running   1/1           1/1      1.24.10+vmware.1   <none>      dev   1.24.10---vmware.1-tkg.1
    mgmt-cluster         tkg-system  running   1/1           1/1      v1.25.7+vmware.1   management  dev   v1.25.7---vmware.1-tkg.1
    
  3. To discover which versions of Kubernetes are made available by a management cluster, run the tanzu kubernetes-release get command:

    tanzu kubernetes-release get
    

    The output lists all of the versions of Kubernetes that you can use to deploy clusters, with the following notes:

    • COMPATIBLE: The current management cluster can deploy workload clusters with this Tanzu Kubernetes release (tkr).
    • UPDATES AVAILABLE: This tkr is not the most current in its Kubernetes version line. Any workload clusters running this tkr version can be upgraded to newer versions.

    For example:

    NAME                       VERSION                  COMPATIBLE  ACTIVE  UPDATES AVAILABLE
    v1.23.16---vmware.1-tkg.1  v1.23.16+vmware.1-tkg.1  True        True
    v1.24.11---vmware.1-tkg.1  v1.24.11+vmware.1-tkg.1  True        True
    v1.25.7---vmware.1-tkg.1   v1.25.7+vmware.1-tkg.1   True        True
    
  4. To discover the newer tkr versions to which you can upgrade a workload cluster running an older tkr version, run the tanzu kubernetes-release available-upgrades get command, specifying the current tkr version:

    tanzu kubernetes-release available-upgrades get 1.22.17---vmware.1-tkg.1
    

    This command lists all of the available Kubernetes versions to which you can upgrade clusters that are running the specified version.

    You can also discover the tkr versions that are available for a specific workload cluster by specifying the cluster name in the tanzu cluster available-upgrades get command:

    tanzu cluster available-upgrades get k8s-1-23-16-cluster
    

    This command lists all of the Kubernetes versions that are compatible with the specified cluster.

    You cannot skip minor versions when upgrading your tkr version. For example, you cannot upgrade a cluster directly from v1.23.x to v1.25.x. You must upgrade a v1.23.x cluster to v1.24.x before upgrading the cluster to v1.25.x.

  5. If the cluster is running on Azure, set the AZURE_CLIENT_SECRET environment variable before upgrading the cluster:

    export AZURE_CLIENT_SECRET=YOUR-AZURE-CLIENT-SECRET
    
  6. Run the tanzu cluster upgrade CLUSTER-NAME command and enter y to confirm. To skip the confirmation step, specify the --yes option.

    To upgrade the cluster to the default version of Kubernetes for this release of Tanzu Kubernetes Grid, run the tanzu cluster upgrade command without any options. In this release, the default version is v1.25.7. For example:

    tanzu cluster upgrade k8s-1-24-10-cluster
    

    If the cluster is not running in the default namespace, specify the --namespace option:

    tanzu cluster upgrade CLUSTER-NAME --namespace NAMESPACE-NAME
    

    If an upgrade times out before it completes, run tanzu cluster upgrade again and specify the --timeout option with a value greater than the default of 30 minutes:

    tanzu cluster upgrade CLUSTER-NAME --timeout 45m0s
    
    Important

    Operations on Azure sometimes take longer than on other platforms. If you are upgrading clusters on Azure, routinely set the --timeout option in order to avoid failures.

    If multiple base VM images in your IaaS account have the same version of Kubernetes that you are upgrading to, use the --os-name option to specify the OS you want. See Selecting an OS During Cluster Upgrade for more information.

    For example, on vSphere if you have uploaded both Photon and Ubuntu OVA templates with Kubernetes v1.25.7, specify --os-name ubuntu to upgrade your workload cluster to run on Ubuntu.

    tanzu cluster upgrade CLUSTER-NAME --os-name ubuntu
    

    Since you cannot skip minor versions of tkr, the upgrade command fails if you try to upgrade a cluster that is more than one minor version behind the default version. For example, you cannot upgrade directly from v1.23.x to v1.25.x. To upgrade a cluster to a version of Kubernetes that is not the default version for this release of Tanzu Kubernetes Grid, specify the --tkr option with the NAME of the chosen version, as listed by tanzu kubernetes-release get above. For example, to upgrade the cluster k8s-1-23-16-cluster from v1.23.16 to v1.24.11.

    tanzu cluster upgrade k8s-1-23-16-cluster --tkr v1.24.11---vmware.1-tkg.1
    
  7. When the upgrade finishes, run the tanzu cluster list command to check that the workload cluster has been upgraded:

    tanzu cluster list --include-management-cluster -A
    
  8. Regenerate the admin kubeconfig:

    tanzu cluster kubeconfig get CLUSTER-NAME --admin
    

    Where CLUSTER-NAME is the name of the workload cluster.

    Important

    If you don’t renew the kubeconfig after upgrading, you won’t be able to access the cluster once it expires.

  9. If you are using an LDAP or OIDC identity provider, confirm that you can authenticate to the cluster with kubectl. For example:

    kubectl get pods -A --kubeconfig my-cluster-credentials
    
  10. Upgrade any CLI-managed packages such as Contour, Fluent Bit or Prometheus that are running on your workload clusters. For more information on upgrading CLI-managed packages, see Update a Package.

    Important

    If you are have installed Prometheus on a workload cluster and you upgrade the workload cluster to Kubernetes v1.25, you must upgrade Prometheus to version 2.37.0+vmware.3-tkg.1. Earlier versions of the Prometheus package, for example version 2.37.0+vmware.1-tkg.1, are not compatible with Kubernetes 1.25.

What to Do Next

You can now continue to use the Tanzu CLI to manage your clusters. For more information, see Creating and Managing TKG 2.2 Workload Clusters with the Tanzu CLI.

check-circle-line exclamation-circle-line close-line
Scroll to top icon