This topic explains how to list available Kubernetes versions and deploy a workload cluster with a non-default Kubernetes version.
Each version of Tanzu Kubernetes Grid with a standalone management cluster provides a default and two non-default versions of Kubernetes, which you can list by running tanzu kubernetes-release get
.
Tanzu Kubernetes Grid manages Kubernetes versions with Tanzu Kubernetes release (TKr) objects. A TKr specifies the OS images, core Kubernetes components, and bootstrap packages that are compatible with the Kubernetes version that is defined in the TKr. When you run tanzu cluster create
with a default or a non-default TKr, Tanzu Kubernetes Grid uses the components and bootstrap packages specified in the TKr to create the cluster. It also reads your cluster configuration file to determine which of the compatible OS images to use when creating the cluster.
For a full list of supported Kubernetes versions, see Supported Kubernetes Versions in Tanzu Kubernetes Grid v2.4 in VMware Tanzu Kubernetes Grid v2.4 Release Notes.
To list all available Kubernetes releases with their current compatibility and upgrade status, run tanzu kubernetes-release get
with an optional version match argument, for example:
tanzu kubernetes-release get
.v1.26.8
, run tanzu kubernetes-release get v1.26.8
.tanzu kubernetes-release get
NAME VERSION COMPATIBLE ACTIVE UPDATES AVAILABLE
v1.25.13---vmware.1-tkg.1 v1.25.13+vmware.1-tkg.1 True True
v1.26.8---vmware.1-tkg.1 v1.26.8+vmware.1-tkg.1 True True
v1.27.5---vmware.1-tkg.1 v1.27.5+vmware.1-tkg.1 True True
To discover the TKr versions that are available for a specific workload cluster, run tanzu cluster available-upgrades get
with the full name of the cluster, for example:
tanzu cluster available-upgrades get my-cluster
A TKr can be activated or deactivated. To activate a TKr:
tanzu kubernetes-release activate TKR-NAME
For example:
tanzu kubernetes-release activate v1.27.5---vmware.1-tkg.1
To deactivate a TKr:
tanzu kubernetes-release deactivate TKR-NAME
For example:
tanzu kubernetes-release deactivate v1.27.5---vmware.1-tkg.1
Each release of Tanzu Kubernetes Grid provides a default version of Kubernetes. The default version for Tanzu Kubernetes Grid v2.4 is Kubernetes v1.27.5.
As upstream Kubernetes releases patches or new versions, VMware publishes them in a public registry and the Tanzu Kubernetes release controller imports them into the management cluster. This lets the Tanzu CLI create clusters based on the new versions.
On vSphere and Azure, you need to take an additional step before you can deploy clusters that run non-default versions of Kubernetes:
Convert your target Kubernetes version listed in the output of the tanzu kubernetes-release get
command into its Azure image SKU as follows:
v
to k8s-
..
to dot
in the version number.+vmware.*
to -ubuntu-2004
, to designate Ubuntu v20.04, the default OS version for all Tanzu Kubernetes Grid VMs on Azure.k8s-1dot27dot5-ubuntu-2004
, k8s-1dot26dot8-ubuntu-2004
.
az vm image terms accept
. For example:az vm image terms accept --publisher vmware-inc --offer tkg-capi-2022-06-24 --plan k8s-1dot27dot5-ubuntu-2004
To deploy the workload cluster running a non-default Kubernetes version:
If you are deploying a plan-based cluster, set an environment variable ALLOW_LEGACY_CLUSTER
to true
:
export ALLOW_LEGACY_CLUSTER=true
If the Kubernetes patch version that you are deploying the cluster with is older than the latest supported patch for its minor version as listed in Supported Kubernetes Versions create a ConfigMap
object for its TKr in the tkg-system
namespace:
Create an ConfigMap
object definition in the tkg-system
namespace that lists the older TKrs:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: tkg-system
name: TKR-CONFIG-NAME
labels:
run.tanzu.vmware.com/additional-compatible-tkrs: ""
data:
tkrVersions: |
- TKR-VERSON-1
- TKR-VERSON-2
Where TKR-CONFIG-NAME
is any legal object name, such as tkg-v2.4.1
, and the TKR-VERSON-*
values are the older TKr versions as listed by tanzu kubernetes-release get
, for example v1.24.11--vmware.1-tkg
.
Create the ConfigMap
object:
kubectl apply -f old-tkrs-config.yaml
To deploy a workload cluster with a version of Kubernetes that is not the default for your version of Tanzu Kubernetes Grid, specify the Tanzu Kubernetes release in the --tkr
option. For example, to deploy a Kubernetes v1.25.13 cluster, run:
tanzu cluster create my-1-25-10-cluster --tkr v1.25.13---vmware.1-tkg
For more details on how to create a workload cluster, see Create Workload Clusters.
For common combinations of OS version, Kubernetes version, and target infrastructure, Tanzu Kubernetes Grid with a standalone management cluster provides default machine images. You can optionally specify custom machine images, including images that you build yourself.
The sections below explain how to deploy a cluster with an alternative or custom-built machine image:
In a cluster configuration file you set OS_NAME
, OS_VERSION
, and OS_ARCH
to specify the base machine image, for example ubuntu
, 20.04
or photon
, 3
; and amd64
. These default to OS images that VMware distributes, but you can also specify your own alternative OS image to use for cluster nodes:
Upload the image to vSphere as described in Import the Base Image Template into vSphere.
Do one of the following, depending on whether your image shares the same OS version as an existing machine image distributed by VMware:
If you are specifying an alternative image with the same OS name, version, and architecture as an existing TKr and image that VMware distributes, edit its OSImage
object, for example:
kubectl edit OSImage v1.26.8---vmware.1-tkg.1-6c92aa1cbb96b644085e6229f41ef14d
Otherwise, create and apply a new OSImage
object, using an existing OSImage
object definition as a template. Set its metadata.name
and spec.image.ref.version
values to the OVA filename.
In the OSImage
object’s spec.image.ref
block, add an identifier field and value. For example, to enable an alternative to the Kubernetes v1.26.8 on Ubuntu 20.04 image that VMware distributes:
apiVersion: run.tanzu.vmware.com/v1alpha3
kind: OSImage
metadata:
creationTimestamp: "2023-06-02T06:08:35Z"
generation: 2
labels:
image-type: ova
os-arch: amd64
os-name: ubuntu
os-type: linux
...
spec:
image:
ref:
MY-FIELD: MY-VALUE
version: v1.26.8---vmware.1-tkg.1-6c92aa1cbb96b644085e6229f41ef14d
type: ova
kubernetesVersion: v1.26.8+vmware.1
os:
arch: amd64
name: ubuntu
type: linux
version: "2004"
Where MY-FIELD
and MY-VALUE
are the identifier field and value that you set to include in Cluster
object specs.
MY-VALUE
must be 0-63 characters long and can be empty, or else it must start and end with alphanumeric characters and contain alphanumerics, dashes (-
), underscores (_
), or dots (.
).OSImage
object’s metadata.labels
settings following the pattern ova-MY-FIELD: MY-VALUE
.To use the alternative machine image in a cluster, edit the Cluster
object to add the setting ova-MY-FIELD=MY-VALUE
to the metadata.annotations
for the cluster nodes that will use the image.
metadata.annotations
under topology.controlPlane
.metadata.annotations
under topology.workers.machineDeployments
for the machine deployments that will use the image.For example:
metadata:
annotations:
run.tanzu.vmware.com/resolve-os-image: image-type=ova,ova-my-field=my-value
You can build your own machine image to use for cluster nodes. Reasons to do this include:
For instructions, see Build Machine Images.