Upgrade Standalone Management Clusters

To upgrade Tanzu Kubernetes Grid with a standalone management cluster, you must first upgrade the standalone management cluster. You cannot upgrade workload clusters until you have upgraded the management cluster that manages them.

If you are running TKG with vSphere with Tanzu Supervisor, you do not follow this procedure. Instead, you upgrade the Supervisor as part of vSphere and update the Supervisor’s Kubernetes version by upgrading its TKrs.

Important
Tanzu Kubernetes Grid v2.4.x is the last version of TKG that supports upgrading existing standalone TKG management clusters on AWS and Azure. The ability to upgrade standalone TKG management clusters on AWS and Azure will be removed in the Tanzu Kubernetes Grid v2.5 release.

Starting from now, VMware recommends that you use Tanzu Mission Control to create native AWS EKS and Azure AKS clusters. However, upgrading existing standalone TKG management clusters on AWS and Azure remains fully supported for all TKG releases up to and including TKG v2.4.x.

For more information, see Deprecation of TKG Management and Workload Clusters on AWS and Azure in the VMware Tanzu Kubernetes Grid v2.4 Release Notes.

Upgrading the management cluster automatically upgrades the auto-managed packages that it runs.

Note
After you have installed the v2.1 CLI but before a standalone management cluster has been upgraded, all context-specific CLI command groups (tanzu cluster, tanzu kubernetes-release) plus all of the management-cluster plugin commands except for tanzu mc upgrade and tanzu mc create are unavailable and not included in Tanzu CLI --help output.

Management clusters and workload clusters use client certificates to authenticate clients. These certificates are valid for one year. To renew them, upgrade your clusters at least once a year or rotate them manually as described in Renew Cluster Certificates (Standalone MC) or the VMware Knowledge Base article How to rotate certificates in a Tanzu Kubernetes Grid cluster.

Prerequisites

You have reviewed the known issues for upgrade. See Known Issues.
- If the Avi Controller might have been rotated since the last management cluster upgrade, follow the steps in Before upgrade, you must manually update a changed Avi certificate in the tkg-system package values.
You performed the steps in Upgrading Tanzu Kubernetes Grid that occur before the step for upgrading management clusters.
If you deployed the previous version of Tanzu Kubernetes Grid in an Internet-restricted environment, you have performed the steps in Prepare an Internet-Restricted Environment to recreate and run the required scripts with the new component image versions.
If you have an existing management cluster of a version up to and including 1.6.0, that is already running on vSphere 7 without a Supervisor Cluster and you want to upgrade vSphere to vSphere 8, upgrade the management cluster and its workload clusters to Tanzu Kubernetes Grid v2.1.x or v1.6.1, ideally before you upgrade from vSphere 7 to vSphere 8. If you are running a standalone management cluster on vSphere 7 and your vSphere instance is upgraded to vSphere 8 before you upgrade to Tanzu Kubernetes Grid v2.1.x or v1.6.1, you must upgrade the management cluster and workload clusters as soon as possible. Tanzu Kubernetes Grid releases up to and including v1.6.0 are not supported on vSphere 8.

Procedure

Run the tanzu login command to see an interactive list of management clusters available for upgrade.
```
tanzu login
```
Select the management cluster that you want to upgrade. See List Management Clusters and Change Context for more information.
Get the admin credentials of the cluster. The Tanzu CLI alias mc is short for management-cluster.
```
tanzu mc kubeconfig get --admin
```

Connect kubectl to the management cluster. 

kubectl config use-context CLUSTER-NAME-admin@CLUSTER-NAME.

If the management cluster is running on Azure, set the AZURE_CLIENT_SECRET environment variable before upgrading the cluster:
```
export AZURE_CLIENT_SECRET=YOUR-AZURE-CLIENT-SECRET
```
If the management cluster was originally created in TKG v1.3 or earlier, or if you are not sure, do the following. These steps add a secret that is needed to upgrade management clusters to their package-based configuration in TKG v2.1, but that was not included in management clusters created before TKG v1.4:
1. Deactivate the package-based-cc feature flag to enable legacy management cluster creation dry-run:
```
tanzu config set features.management-cluster.package-based-cc false
```
2. Obtain a more up-to-date legacy-style manifest for the management cluster by dry-running management cluster creation with its original configuration file. For example on vSphere:
```
tanzu mc create -f mgmt-config.yaml --dry-run > legacy-mgmt.yaml
Do you want to configure vSphere with Tanzu? [y/N]: N
Would you like to deploy a non-integrated Tanzu Kubernetes Grid management cluster on vSphere 7.0? [y/N]: y
```
3. Extract the needed -config-values secret:
```
cat legacy-mgmt.yaml | yq 'select(.metadata.name == "MANAGEMENT-CLUSTER-NAME-config-values")' > mgmt-config-secret.yaml
```
4. In the management cluster context, apply the secret to the management cluster:
```
kubectl apply -f mgmt-config-secret.yaml
```
5. Re-enable the package-based-cc feature flag, to disallow legacy management cluster creation.
```
tanzu config set features.management-cluster.package-based-cc true
```
Run the tanzu mc upgrade command and enter y to confirm.

Note
After you run this command, non-admin users cannot log in to the associated workload clusters until the Pinniped pods finish restarting.
```
tanzu mc upgrade
```
If multiple base VM images in your IaaS account have the same version of Kubernetes that you are upgrading to, use the --os-name option to specify the OS you want. See Selecting an OS During Cluster Upgrade for more information.

For example, on vSphere if you have uploaded both Photon and Ubuntu OVA templates with Kubernetes v1.24.10, specify --os-name ubuntu to upgrade your management cluster to run on an Ubuntu VM.
```
tanzu mc upgrade --os-name ubuntu
```
To skip the confirmation step when you upgrade a cluster, specify the --yes option.
```
tanzu mc upgrade --yes
```
The upgrade process first upgrades the Cluster API providers for vSphere, Amazon Web Services (AWS), or Azure that are running in the management cluster. Then, it upgrades the version of Kubernetes in all of the control plane and worker nodes of the management cluster.

Important
While a management cluster is upgrading, do not run tanzu cluster or tanzu mc commands against it or the workload clusters that it manages, for example from another bootstrap machine or shell window.

If the upgrade times out before it completes, run tanzu mc upgrade again and specify the --timeout option with a value greater than the default of 30 minutes.
```
tanzu mc upgrade --timeout 45m0s
```
Note
After you have installed the v2.1 CLI but before a management cluster has been upgraded, all context-specific CLI command groups (tanzu cluster, tanzu kubernetes-release) plus all of the management-cluster plugin commands except for tanzu mc upgrade and tanzu mc create are unavailable and not included in Tanzu CLI --help output.

After the upgrade finishes, run the tanzu cluster list command with the --include-management-cluster -A options again to check that the management cluster has been upgraded.

tanzu cluster list --include-management-cluster -A

You see that the management cluster is now running the new version of Kubernetes, but that the workload clusters are still running previous versions of Kubernetes.

 NAME                 NAMESPACE   STATUS    CONTROLPLANE  WORKERS  KUBERNETES         ROLES       PLA   TKR
 k8s-1-22-13-cluster  default     running   1/1           1/1      v1.22.13+vmware.1  <none>      dev   v1.22.13---vmware.1-tkg.1
 k8s-1-23-10-cluster  default     running   1/1           1/1      v1.23.10+vmware.1  <none>      dev   v1.23.10---vmware.1-tkg.1
 mgmt-cluster         tkg-system  running   1/1           1/1      v1.24.10+vmware.1  management  dev   v1.24.10---vmware.1-tkg.1

Regenerate the admin kubeconfig:
```
tanzu management-cluster kubeconfig get --admin
```
The following is sample output of the command:
```
Credentials of cluster 'mgmt' have been saved
You can now access the cluster by running 'kubectl config use-context mgmt-admin@mgmt'
```
Important
If you don’t renew the kubeconfig after upgrading, you won’t be able to access the cluster once it expires.

What to Do Next

You can now:

Upgrade the workload clusters that this management cluster manages.
Create new workload clusters. By default, any new clusters that you deploy with this management cluster will run the new default version of Kubernetes. However, if required, you can use the tanzu cluster create command with the --tkr option to deploy new clusters that run different versions of Kubernetes. For more information, see Multiple Kubernetes Versions.