Update the Supervisor

Learn how to update your Supervisors, including the Kubernetes version that the Supervisors are running. The Supervisors in your environment must always run a supported Kubernetes version.

There is a version entity for vSphere IaaS control plane. The version entity is a semantic version string in the form v1.28.3+vmware.wcp.1-vsc0.1.9-23708114, where the prefix is the Kubernetes version (v1.28.3) and the suffix is the Supervisor release version released with the corresponding vCenter Server version and build number. (vvsc0.1.9-23708114).

When you upgrade vCenter Server to a new version, the vSphere Namespaces version is also updated. Each vSphere Namespaces version contains a new Supervisor version and two earlier supported versions.

vSphere Namespaces version 0.1.9, which is delivered with vCenter Server 8 Update 3 includes three Supervisor versions:

• v1.26.8+vmware.wcp.1-vsc0.1.9-23708114
• v1.27.5+vmware.wcp.1-vsc0.1.9-23708114
• v1.28.3+vmware.wcp.1-vsc0.1.9-23708114

Note: To use Tanzu Kubernetes Grid 3.0, you must update the Supervisor to one of the three supported versions included with vSphere Namespaces 0.1.9.

.

Prerequisites

• Check the supported Kubernetes versions for the Supervisor in the vSphere IaaS control plane release notes. Each vCenter Server release contains three Kubernetes versions for the Supervisor - the most recent Kubernetes version for theSupervisor released with the corresponding vCenter Server version, and two previous versions, which are supported for at least 12 months.
• Install the currently supported Kubernetes versions for the Supervisor by upgrading the vCenter Server appliance to the vCenter Server version that provides them. See Upgrading the vCenter Server Appliance.

Note: When you update the Supervisor, all provisioned Tanzu Kubernetes Grid clusters must be using a TKr version compatible with the new Supervisor K8s version in vSphere 8 Update 3. Since Tanzu Kubernetes Grid becomes an independent Supervisor Service starting from vSphere Update 3, the TKr versions depend on the Tanzu Kubernetes Grid service versions moving forward.

Note: Updating a Supervisor can trigger a rolling update of the Tanzu Kubernetes clusters deployed there. See Understanding the Rolling Update Model for TKG Service Clusters.

Procedure

1. Log in to the vCenter Server as a vSphere administrator.
2. Select Menu > Workload Management.
3. Select the Namespaces > Updates tab.
4. Select the Available Version that you want to update to.
   For example, select the version vv1.28.3+vmware.wcp.1-vsc0.1.9-23708114.
   Note: You must update incrementally. You cannot skip updates, such as from 1.26 to 1.28. The path should be 1.26, 1.27, 1.28.
5. Select a Supervisor to update.
6. Click Apply Updates.
   The system runs a series of pre-checks to verify the compatibility of the different components against the Supervisor Kubernetes version to which you want to update. When the pre-checks are completed successfully, you can update the Supervisor.

What are Supervisor Update Pre-checks and How to Troubleshoot Them

Learn which are the pre-checks that are run before updating the Supervisor and how to troubleshoot in case of errors resulting from failed pre-checks.

The Supervisor and the workloads that it runs interact with multiple vSphere resources (storage, memory, CPU, and network) running on different layers of vSphere. As a vSphere administrator, when you initiate a Supervisor update, the system runs pre-checks to verify that workload compatibility and resource requirements are met. These pre-checks include health checks, resource checks, and dependency checks on different software components to ensure that a successful Supervisor update is possible.

Supervisor Update Pre-checks and Troubleshooting Errors

Pre-check	Description	Error	Troubleshooting
Datastore requirement verified	Checks if the datastore has enough resource to create the Supervisor control plane VMs.	Not enough free space on chosen datastore to deploy one more Supervisor CPVM. Need at least <value>.	Reclaim the unused space from the configured datastores for Supervisor control plane VMs or add additional datastores to the control plane storage policy.
Supervisor Control Plane VMs health verified	Checks if the health of the target Supervisor control plane VM is good enough.	Cluster <Supervisor> is unhealthy: <error message>	Depending on the reported error, follow the resolutions at Resolving Errors Health Statuses on a Supervisor Control Plane VMs During Activation Or Update
Etcd cluster health verified	Check if the etcd cluster of the current Supervisor has a good health status	Etcd members are not healthy during cluster upgrade pre-check.	Follow the steps in Tanzu Kubernetes Cluster Upgrade Stuck - EtcdMemberHealthy condition is unknown
Network compatibility verified	Checks if the NSX version is compatible with NSX Container Plug-in (NCP) version.	NSX version <version> is not compatible with NCP version <version>. Please choose NSX version from compatible list <list> but not in incompatible list <list>.	Make sure that the NSX and NCP versions are compatible.
Workload cluster(s) compatibility verified	Checks if running TKG clusters are going to be incompatible after upgrade	Proposed upgrade to Namespaces cluster <Supervisor> is incompatible with one or more Tanzu Kubernetes cluster(s) Note: The error message for TKG clusters differ in format based on the source Supervisor from which the upgrade is performed. BD	Update the TKr version of the incompatible TKG clusters to a version that is compatible with the new Kubernetes version of the Supervisor. See . This pre-check only occurs only during the first Supervisor update in vSphere 8 Update 3.
Supervisor Service compatibility verified	Checks if the versions of the installed Supervisor Services are compatible with the target Supervisor Kubernetes version.	The incoming Supervisor requires the Supervisor Service <service name> version <number> but the installed Service is on version <number>. Upgrading the Supervisor will make the installed service incompatible. The Supervisor Service <service name> version <number> requires Supervisor release version <number> but the Supervisor will be on version <number>. Upgrading the Supervisor will make the installed service incompatible.	Update the versions of the incompatible Supervisor Services.

Resolving Errors Health Statuses on a Supervisor Control Plane VMs During Activation Or Update

After you activate a Supervisor, update the Supervisor Kubernetes version, or edit the settings of an existing Supervisor, all the settings that you have specified are validated and applied to the Supervisor until the configuration completes. Health checks are performed on the entered parameters that might detect errors in the configuration resulting in an error health status of the Supervisor. You must resolve these error health statuses so that the configuration or update of the Supervisor is possible.

Table 1. vCenter Server Connection Errors

Error Message	Cause	Solution
Unable to resolve the vCenter Primary Network Identifier <FQDN> with the configured management DNS server(s) on control plane VM <VM name>. Validate that the management DNS servers <server name> can resolve <network name>.	At least one management DNS server is reachable. At least one management DNS is statically supplied. The management DNS servers do not have any host name lookup for the vCenter Server PNID. The vCenter Server PNID is a domain name, not a static IP address.	Add a host entry for the vCenter Server PNID to the management DNS servers. Verify the configured DNS servers are correct.
Unable to resolve the vCenter Primary Network Identifier <network name> with the DNS server(s) acquired via DHCP on the management network of the control plane VM <VM name>. Validate that the management DNS servers can resolve <network name>.	The management DNS servers supplied by the DHCP server (at least one) are reachable. The management DNS servers are statically supplied. The management DNS servers do not have any host name lookup for the vCenter Server PNID. The management DNS servers do not have any host name lookup for the vCenter Server PNID. The vCenter Server PNID is a domain name, not a static IP address.	Add a host entry for the vCenter Server PNID to the management DNS servers supplied by the configured DHCP server. Verify the DNS servers supplied by the DHCP server are correct.
Unable to resolve the host <host name> on control plane VM <VM name> , as there are no configured management DNS servers.	The vCenter Server PNID is a domain name, not a static IP address. There are no DNS servers configured.	Configure a management DNS server.
Unable to resolve the host <host name> on control plane VM <VM name>. The hostname ends with the '.local' top level domain, which requires 'local' to be included in the management DNS search domains.	The vCenter Server PNID contains .local as a top-level domain (TLD), but the configured search domains do not includelocal.	Add local to the management DNS search domains.
Unable to connect to the management DNS servers <server name> from control plane VM <VM name>. The connection was attempted over the workload network.	The management DNS servers are unable to be connected to vCenter Server. The provided worker_dns values wholly contain the provided management DNS values. This means that traffic is routed via the workload network, as the Supervisor must pick one network interface to direct static traffic to these IPs.	Check the Workload Network to verify that it can route to the configured management DNS servers. Verify there are no conflicting IP addresses that might trigger alternate routing between the DNS servers and some other servers on the Workload Network. Verify the configured DNS server is, in fact, a DNS server, and is hosting its DNS port on port 53. Verify the workload DNS servers are configured to allow connections from the IPs of the control plane VMs (the Workload Network provided IPs). Verify that there are no typos in the management DNS servers' addresses. Verify search domains don't include an unnecessary '~' that could be resolving the host name incorrectly.
Unable to connect to the management DNS servers <server name> from the control plane VM <VM name>.	Unable to connect to the DNS servers.	Check the management network to verify that routes to the management DNS servers exist. Verify there are no conflicting IP addresses that may trigger alternate routing between the DNS servers and other servers. Verify the configured DNS server is, in fact, a DNS server, and is hosting its DNS port on port 53. Verify the management DNS servers are configured to allow connections from the IPs of the control plane VMs. Verify that there are no typos in the management DNS servers' addresses. Verify that search domains do not include an unnecessary '~' that could be resolving the host name incorrectly.
Unable to connect to <component name> <component address> from control plane VM <vm name>. Error: error message text	A generic network failure occurred. Error occurred while connecting to actual connecting to vCenter Server.	Validate that the host name or IP address of the configured components, such as vCenter Server, HAProxy, NSX Manager, or NSX Advanced Load Balancer are correct. Validate any external network settings such as conflicting IPs, firewall rules, and others, on the management network.
The control plane VM <VM name> was unable to validate the vCenter <vCenter Server name> certificate. The vCenter server certificate is invalid.	The certificate provided byvCenter Server is in invalid format, and therefore is untrusted.	Restart wcpsvc to verify that the Trusted Roots bundle in the control plane VMs are up-to-date with the latest vCenter Server root certificates. Verify that the vCenter Servercertificate is actually a valid certificate.
The control plane VM <VM name> does not trust the vCenter <vCenter Server name>certificate.	The vmca.pem certificate presented by vCenter Server is different from what is configured to the control plane VMs. The trusted root certificates were replaced in the vCenter Server appliance, but wcpsvc wasn't restarted.	Restart wcpsvc to verify that the Trusted Roots bundle in the control plane VMs are up-to-date with the latest vCenter Server certificate roots.

Table 2. NSX Manager Connection Errors

The control plane VM <VM name> was unable to validate the NSX Server<NSX server name> certificate. The thumbprint returned by the server <NSX-T address> doesn't match the expected client certificate thumbprint registered in vCenter <vCenter Server name>	The SSL thumbprints registered to the Supervisor don't match the SHA-1 hash of the certificate presented by the NSX manager.	Re-enable trust on the NSX manager between NSX and thevCenter Server instance. Restart wcpsvc on vCenter Server.
Unable to connect to <component name> <component address> from control plane VM <vm name>. Error: error message text	A generic network failure occurred.	Validate any external network settings,conflicting IPs, firewall rules, and others, on the management network for the NSX manager. Verify the NSX manager IP in the NSX extension is correct. Verify that the NSX manager is running.

Table 3. Load Balancer Errors

The control plane VM <vm name> does not trust the load balancer's (<load balancer>- <load balancer endpoint>) certificate.	The certificate the load balancer presents is different from the certificate that is configured to the control plane VMs.	Verify that you have configured the correct Management TLS certificate to the load balancer.
The control plane VM <vm name> was unable to validate the load balancer's (<load balancer>- <load balancer endpoint> certificate. The certificate is invalid.	The certificate the load balancer presents is in an invalid format, or expired.	Correct the server certificate of the configured load balancer.
The control plane VM <vm name> was unable to authenticate to the load balancer (<load balancer>- <load balancer endpoint> with the username <user name> and the supplied password.	The user name or password of the load balancer are incorrect.	Verify the if the user name and password configured to the load balancer are correct.
An HTTP error occurred when attempting to connect to the load balancer (<load balancer>- <load balancer endpoint> from the control plane VM <vm name>.	The control plane VMs can connect to the load balancer endpoint, but the endpoint does not return a successful (200) http response	Verify that the load balancer is healthy and accepting requests.
Unable to connect to <load balancer> (<load balancer endpoint>) from control plane VM <vm name>. Error: <error text>	A generic network failure occurred. Typically, it means the load balancer is not working, or some firewall blocks the connection.	Validate that the load balancer endpoint is accessible Validate no firewalls are blocking the connection to the load balancer.