Upgrading Tanzu Kubernetes Grid Integrated Edition (Antrea Networking)

This topic explains how to upgrade VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) in Antrea Networking environments from v1.18 to v1.19 on vSphere, Amazon Web Services (AWS), and Azure.

For instructions on upgrading Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX networking, see Upgrading Tanzu Kubernetes Grid Integrated Edition (NSX Networking).

Warning: Do not manually upgrade your Kubernetes version. Tanzu Kubernetes Grid Integrated Edition includes the compatible Kubernetes version.

Overview

Before you upgrade, follow the procedures in Prepare to Upgrade below to plan and prepare your upgrade.

After you complete the preparation steps, continue to the procedures in Perform the Upgrade below. These steps guide you through the process of upgrading VMware Tanzu Operations Manager (Ops Manager) and the Tanzu Kubernetes Grid Integrated Edition tile, importing a new stemcell, and applying the changes to your deployment.

After you complete the upgrade, follow the procedures in After the Upgrade below to verify that your upgraded Tanzu Kubernetes Grid Integrated Edition deployment is running properly.

Prepare to Upgrade

• If you have not already, complete all of the steps in Upgrade Preparation Checklist for.
• To upgrade with multiple datacenters, see below.
  • You must use Ops Manager. You cannot upgrade on multiple datacenters using the Management Console.

Note: TKGI v1.19 does not support Flannel CNI. If you have clusters running with Flannel CNI on TKGI 1.18 or earlier, create new clusters with Antrea CNI and move all workloads from Flannel clusters to Antrea clusters before you upgrade TKGI to 1.19.

Prepare to Upgrade with Multiple Datacenters

To upgrade a TKGI installation that runs on multiple datacenters, include the following steps:

1. Upload to Ops Manager a TKGI tile version that supports multiple datacenters.
   • Support for multiple datacenters varies between TKGI patch releases within the same minor release line, with later patch releases supporting the functionality while earlier ones do not.
2. In the TKGI tile > Kubernetes Cloud Profile pane, set vSphere Datacenter to a comma-separated list of datacenters, for example "DC1, DC2, folder-name/DC3". For datacenters that are under folders in your inventory, include the folder in the name.

3. If you are using NSX networking, update the TKGI tile > Networking pane > vSphere Cluster list to include the clusters’ datacenters in their names, for example, "DC1:cluster1, DC2:cluster2, DC-folder-name/DC3:cluster3".

   After upgrading TKGI with multiple datacenters, you can use the tkgi CLI to upgrade TKGI clusters.

   Note: The TKGI Management Console does not yet support upgrading TKGI running on multiple datacenters.

Perform the Upgrade

This section describes the steps required to upgrade to Tanzu Kubernetes Grid Integrated Edition v1.19:

1. Upgrade Ops Manager
2. Download and Import Tanzu Kubernetes Grid Integrated Edition v1.19
3. Download and Import Stemcells
4. Modify Plan CNI Configuration
5. Verify Errand Configuration
6. Verify Other Configurations
7. Apply Changes to the Tanzu Kubernetes Grid Integrated Edition Tile

Upgrade Ops Manager

Each version of Tanzu Kubernetes Grid Integrated Edition is compatible with multiple versions of Ops Manager.

Warning: If you use an automated pipeline to upgrade TKGI, see Configure Automated Ops Manager and Ubuntu Jammy Stemcell for VMware Tanzu Downloading in Configuring the Upgrade Pipeline.

To determine Ops Manager compatibility and, if necessary, upgrade Ops Manager:

1. See Broadcom Support to determine if your Ops Manager version is compatible with Tanzu Kubernetes Grid Integrated Edition v1.19.
2. If your Ops Manager version is not compatible with Tanzu Kubernetes Grid Integrated Edition v1.19, follow the steps below.
3. Upgrade Ops Manager. For instructions, see Import Installation to Ops Manager v3.0 VM in Upgrading Ops Manager in the Ops Manager documentation.
   
4. Verify that the Tanzu Kubernetes Grid Integrated Edition control plane remains functional by performing the following steps:
   1. Add more workloads and create an additional cluster. For more information, see About Cluster Upgrades in Maintaining Workload Uptime and Creating Clusters.
   2. Monitor the Tanzu Kubernetes Grid Integrated Edition control plane in the Tanzu Kubernetes Grid Integrated Edition tile > Status tab. Review the load and resource usage data for the TKGI API and TKGI Database VMs. If any levels are at capacity, scale up the VMs.
      
      

Download and Import Tanzu Kubernetes Grid Integrated Edition v1.19

When you upgrade Tanzu Kubernetes Grid Integrated Edition, your configuration settings typically migrate to the new version automatically. To download and import a Tanzu Kubernetes Grid Integrated Edition version:

1. Download the desired version of the product from Broadcom Support.

2. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

3. Under the Import a Product button, click + next to Tanzu Kubernetes Grid Integrated Edition. This adds the tile to your staging area.

Download and Import Stemcells

TKGI requires an Ubuntu Jammy Stemcell for VMware Tanzu. A Windows 2019 Windows Stemcell for VMware Tanzu is also required if you intend to create Windows worker-based clusters. For information about Windows stemcells, see Configuring Windows Worker-Based Clusters.

Warning: If you use an automated pipeline to upgrade TKGI, see Configure Automated Ops Manager and Ubuntu Jammy Stemcell Downloading in Configuring the Upgrade Pipeline.

If Ops Manager does not have the Ubuntu Jammy Stemcell for VMware Tanzu required for Tanzu Kubernetes Grid Integrated Edition v1.19, the Tanzu Kubernetes Grid Integrated Edition tile displays the message Missing stemcell. To download and import a new Ubuntu Jammy Stemcell for VMware Tanzu, follow the steps below:

1. On the Tanzu Kubernetes Grid Integrated Edition tile, click the Missing stemcell link.

   

2. In the Stemcell Library, locate the Tanzu Kubernetes Grid Integrated Edition tile and note the required stemcell version.

3. Navigate to the Stemcells (Ubuntu Jammy) page on Broadcom Support and download the required stemcell version for your IaaS.

4. Return to the Installation Dashboard in Ops Manager and click Stemcell Library.

5. On the Stemcell Library page, click Import Stemcell and select the stemcell file you downloaded from Broadcom Support.

6. Select the Tanzu Kubernetes Grid Integrated Edition tile and click Apply Stemcell to Products.

7. Verify that Ops Manager successfully applied the stemcell. The stemcell version you imported and applied appears in the Staged column for Tanzu Kubernetes Grid Integrated Edition.

8. Return to the Installation Dashboard.

Modify Container Network Interface Configuration

Tanzu Kubernetes Grid Integrated Edition supports using the Antrea Container Network Interface (CNI) as the CNI for new TKGI-provisioned clusters.

To configure Tanzu Kubernetes Grid Integrated Edition to use Antrea as the CNI for new clusters:

1. In the Installation Dashboard, click Networking.
2. Under Container Networking Interface, select Antrea.
3. Confirm the remaining Container Networking Interface settings.
4. Click Save.

Verify Errand Configuration

To verify your Errands pane is correctly configured, do the following:

1. In the Tanzu Kubernetes Grid Integrated Edition tile, click Errands.

2. Under Post-Deploy Errands:

   • Review the Upgrade all clusters errand:
     • If you want to upgrade the Tanzu Kubernetes Grid Integrated Edition tile and all your existing Kubernetes clusters simultaneously, confirm that Upgrade all clusters errand is set to Default (On). The errand upgrades all clusters. Upgrading Tanzu Kubernetes Grid Integrated Edition-provisioned Kubernetes clusters can temporarily interrupt the service as described in Service Interruptions.
     • If you want to upgrade the Tanzu Kubernetes Grid Integrated Edition tile only and then upgrade your existing Kubernetes clusters separately, deactivate Upgrade all clusters errand. For more information, see Upgrading Clusters.

       Warning: Deactivating the Upgrade all clusters errand causes the TKGI version tagged in your Kubernetes clusters to fall behind the Tanzu Kubernetes Grid Integrated Edition tile version. If you deactivate the Upgrade all clusters errand when upgrading the Tanzu Kubernetes Grid Integrated Edition tile, you must upgrade all your Kubernetes clusters before the next Tanzu Kubernetes Grid Integrated Edition upgrade.

   • Configure the Run smoke tests errand:

     • Set the Run smoke tests errand to On. The errand uses the Tanzu Kubernetes Grid Integrated Edition Command Line Interface (TKGI CLI) to create a Kubernetes cluster and then delete it. If the creation or deletion fails, the errand fails and the installation of the Tanzu Kubernetes Grid Integrated Edition tile is aborted.

3. Click Save.

Verify Other Configurations

To confirm your other Tanzu Kubernetes Grid Integrated Edition tile panes are correctly configured, do the following:

1. Review the Assign AZs and Networks pane.

   Note: When you upgrade Tanzu Kubernetes Grid Integrated Edition, you must place singleton jobs in the AZ you selected when you first installed the Tanzu Kubernetes Grid Integrated Edition tile. You cannot move singleton jobs to another AZ.

2. Review the other configuration panes.
3. Make changes where necessary.

   WARNING: Do not change the number of control plane/etcd nodes for any plan that was used to create currently-running clusters. Tanzu Kubernetes Grid Integrated Edition does not support changing the number of control plane/etcd nodes for plans with existing clusters.

4. Click Save on any panes where you make changes.

Apply Changes to the Tanzu Kubernetes Grid Integrated Edition Tile

To complete the upgrade of the Tanzu Kubernetes Grid Integrated Edition tile:

1. Return to the Installation Dashboard in Ops Manager.

2. Click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.

3. Click Apply Changes.

4. (Optional) To monitor the progress of the Upgrade all clusters errand using the BOSH CLI, do the following:

   1. Log in to the BOSH Director by running bosh -e MY-ENVIRONMENT log-in from a VM that can access your Tanzu Kubernetes Grid Integrated Edition deployment. For more information, see Using BOSH Diagnostic Commands in Tanzu Kubernetes Grid Integrated Edition.
   2. Run bosh -e MY-ENVIRONMENT tasks.
   3. Locate the task number for the errand in the # column of the BOSH output.
   4. Run bosh task TASK-NUMBER, replacing TASK-NUMBER with the task number you located in the previous step.

After the Upgrade

After you complete the upgrade to Tanzu Kubernetes Grid Integrated Edition v1.19, complete the following verifications and upgrades:

• Upgrade the TKGI and Kubernetes CLIs
• Verify the Upgrade

Upgrade the TKGI and Kubernetes CLIs

Upgrade the TKGI and Kubernetes CLIs on any local machine where you run commands that interact with your upgraded version of Tanzu Kubernetes Grid Integrated Edition.

To upgrade the CLIs, download and re-install the TKGI and Kubernetes CLI distributions that are provided with Tanzu Kubernetes Grid Integrated Edition on Broadcom Support.

For more information about installing the CLIs, see the following topics:

• Installing the TKGI CLI

• Installing the Kubernetes CLI

Verify the Upgrade

After you apply changes to the Tanzu Kubernetes Grid Integrated Edition tile and the upgrade is complete, do the following:

1. Verify that your Kubernetes environment is healthy. To verify the health of your Kubernetes environment, see Verifying Deployment Health.

   For any cluster upgrade that fails, you can use the BOSH ID of the upgrade task for debugging. To retrieve the BOSH task ID, see Retrieve Cluster Upgrade Task ID in Verifying Deployment Health.

2. Verify that the Tanzu Kubernetes Grid Integrated Edition control plane remains functional by performing the following steps:
   1. Add more workloads and create an additional cluster. For more information, see About Cluster Upgrades in Maintaining Workload Uptime and Creating Clusters.
   2. Monitor the Tanzu Kubernetes Grid Integrated Edition control plane in the Tanzu Kubernetes Grid Integrated Edition tile > Status tab. Review the load and resource usage data for the TKGI API and TKGI Database VMs. If any levels are at capacity, scale up the VMs.