VMware Tanzu Kubernetes Grid | 06 AUG 2020 | CLI build 16711757 | Component build 16699823

Check for additions and updates to these release notes.

About VMware Tanzu Kubernetes Grid

VMware Tanzu Kubernetes Grid provides Enterprise organizations with a consistent, upstream compatible, regional Kubernetes substrate across SDDC, Public Cloud, and Edge environments that is ready for end-user workloads and ecosystem integrations. TKG builds on trusted upstream and community projects and delivers an engineered and supported Kubernetes platform for end users and partners.

Key features include:

  • The Tanzu Kubernetes Grid installer interface, a graphical installer that walks you through the process of deploying management clusters to either vSphere or Amazon EC2.
  • The Tanzu Kubernetes Grid CLI, providing simple commands that allow you to deploy CNCF conformant Kubernetes clusters to either vSphere or Amazon EC2.
  • Binaries for Kubernetes and all of the components that you need in order to easily stand up an enterprise-class Kubernetes development environment. All binaries are tested and signed by VMware.
  • Extensions for your Tanzu Kubernetes Grid instance, that provide authentication and authorization, logging, networking, and ingress control. 
  • VMware support for your Tanzu Kubernetes Grid deployments.

New Features in Tanzu Kubernetes Grid 1.1.3

Tanzu Kubernetes Grid 1.1.3 is a patch release that includes support for new Kubernetes versions and critical bug fixes.

  • New Kubernetes versions:
    • 1.18.6
    • 1.17.9
  • Cluster API Provider vSphere v0.6.6
  • New vSphere container storage interface image that includes NFS Utils

Behavior Changes Between Tanzu Kubernetes Grid 1.1.2 and 1.1.3

Tanzu Kubernetes Grid v1.1.3 introduces no new behavior compared with v1.1.2.

Supported Kubernetes Versions in Tanzu Kubernetes Grid 1.1.3

Tanzu Kubernetes Grid 1.1.3 provides support for Kubernetes 1.18.6 and 1.17.9. This version also supports the versions of Kubernetes from previous versions of Tanzu Kubernetes Grid.

Tanzu Kubernetes Grid Version Provided Kubernetes Versions Supported in v1.1.3?
1.1.3 1.18.6
1.1.2 1.18.3
1.1.0 1.18.2 YES
1.0.0 1.17.3 YES

Supported Upgrade Paths

You can upgrade Tanzu Kubernetes Grid v1.0.0, v1.1.0, and v1.1.2 to version 1.1.3.

Supported AWS Regions

You can use Tanzu Kubernetes Grid 1.1.3 to deploy clusters to the following AWS regions:

  • ap-northeast-1
  • ap-northeast-2
  • ap-south-1
  • ap-southeast-1
  • ap-southeast-2
  • eu-central-1
  • eu-west-1
  • eu-west-2
  • eu-west-3
  • sa-east-1
  • us-east-1
  • us-east-2
  • us-gov-east-1
  • us-gov-west-1
  • us-west-2

User Documentation

The Tanzu Kubernetes Grid 1.1 documentation applies to all of the 1.1.x releases. It includes information about the following subjects:

Component Versions

The Tanzu Kubernetes Grid 1.1.3 release ships with the following software components:

  • calico_all: v3.11.2+vmware.1
  • cloud_provider_vsphere: v1.1.0+vmware.3
  • cluster_api: v0.3.6+vmware.2
  • cluster_api_aws: v0.5.4+vmware.2
  • cluster_api_vsphere: v0.6.6+vmware.2
  • cni_plugins: v0.8.6+vmware.2
  • containerd: v1.3.4+vmware.1
  • contour: v1.2.1+vmware.1
  • coredns: v1.6.7+vmware.2
  • crash-diagnostics: v0.2.2+vmware.3
  • cri_tools: v1.17.0+vmware.2
  • csi_attacher: v2.0.0+vmware.2
  • csi_livenessprobe: v1.1.0+vmware.8
  • csi_node_driver_registrar: v1.2.0+vmware.2
  • csi_provisioner: v1.4.0+vmware.3
  • dex: v2.22.0+vmware.1
  • envoy: v1.13.1+vmware.1
  • etcd: v3.4.3+vmware.6
  • fluent-bit: v1.3.8+vmware.1
  • gangway: v3.2.0+vmware.1
  • haproxy_ova: v1.2.4+vmware.1
  • jetstack_cert-manager: v0.11.0+vmware.1
  • kube_rbac_proxy: v0.4.1+vmware.2
  • kubernetes: v1.18.6+vmware.1
  • kubernetes-sigs_kind: v0.8.1-1.18.6+vmware.1
  • node_ova: v1.18.6+vmware.1,v1.17.9+vmware.1
  • tanzu_tkg-cli: v1.1.3+vmware.1
  • tkg_extensions_manifests: v1.1.0+vmware.1
  • vsphere_csi_driver: v2.0.0+vmware.3

After you install the Tanzu Kubernetes Grid CLI and run any tkg command, you can see the list of versions for all of the components that ship with Tanzu Kubernetes Grid 1.1.3 in the file ~/.tkg/bom/bom-1.1.3+vmware.1.yaml.

Resolved Issues

  • Cluster API Provider vSphere stops unexpectedly when the VirtualMachine configuration is unavailable

    If Cluster API Provider vSphere cannot read the UUID of a virtual machine it stops unexpectedly because the error is not detected. For more information, see https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/issues/944

  • Installation in Internet-restricted environments fails due to incorrect version for Cluster API Provider AWS

    If you are deploying Tanzu Kubernetes Grid in an internet-restricted environment, the deployment fails because the script that populates your local registry includes the incorrect version of the Cluster API Provider AWS.

  • NFS Utils is missing from vSphere CSI driver

    If you are using VSAN storage on vSphere, when you create a pod, attempting to mount volumes fails with the error: an error occurred during FIP allocation. This occurs because NFS utils is missing from the vsphere-block-csi-driver:v2.0.0_vmware.1 component.

Known Issues

The known issues are grouped as follows.

vSphere Issues
  • Cluster networking issues after upgrading virtual hardware in Base OS templates or cluster VMs

    Due to a kernel issue in Photon OS, if you are using the Calico CNI, the virtual hardware of the photon-3-kube-v1.1.8.6+vmware.1 and photon-3-kube-v.1.17.9+vmware.1 templates, as well as any cluster VMs that you deploy from the templates, must remain at version 13. Do not upgrade these templates or VMs to a virtual hardware version above version 13.


    Use SSH to log in to each VM in the cluster, and run the following command:

    ethtool eth0 -K tx-off

    To make the workaround persist following reboots, create a file at /etc/udev/rules.d/90-netif-disable-hw-offload.rules and paste the following contents into it.

    ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth*", TAG+="netif_hw_tx_offload_disable"	
    ACTION=="add", SUBSYSTEM=="net", KERNEL=="en*", TAG+="netif_hw_tx_offload_disable"
    TAG=="netif_hw_tx_offload_disable", RUN+="/usr/sbin/ethtool -K $name tx off"
  • Cannot log back in to vSphere 7 Supervisor Cluster after connection expires

    When you use kubectl vsphere login to log in to a vSphere 7 Supervisor Cluster, the kubeconfig file that is generated expires after 10 hours. If you attempt to run Tanzu Kubernetes Grid CLI commands against the Supervisor Cluster after 10 hours have passed, you are no longer authorized to do so. If you use kubectl vsphere login to log in to the Supervisor Cluster again, get the new kubeconfig, and attempt to run  tkg add management-cluster cluster_name to add the new kubeconfig to .tkg/config, it throws an error:
    Error: : cannot save management cluster context to kubeconfig: management cluster with context already exists


    1. Set an environment variable for kubeconfig: export KUBECONFIG=$HOME/.kube-tkg/config
    2. Run kubectl vsphere login.

    This updates the Tanzu Kubernetes Grid management cluster kubeconfig for vSphere 7 with Kubernetes without requiring you to update it by using the tkg add management-cluster command.

  • Upgrade to 1.1 fails if the location of the management cluster has changed

    If the location of the management cluster changed after initial deployment, for example because the cluster was renamed, upgrading the management cluster to version 1.1 fails with an error similar to the following:

    "error"="failed to reconcile VM: unable to get resource pool for management-cluster"

    Workaround: Do not change the name of the cluster on which you deploy management clusters.

  • Management cluster deployment fails if the vCenter Server FQDN includes uppercase characters

    If you set the VSPHERE_SERVER parameter in the config.yaml file with a vCenter Server FQDN that includes upper-case letters, deployment of the management cluster fails with the error Credentials not found.

    Workaround: Use all lower-case letters when you specify a vCenter Server FQDN in the config.yaml file.

Upgrade Issues
  • List of clusters shows incorrect Kubernetes version after unsuccessful upgrade attempt

    If you attempt to upgrade a Tanzu Kubernetes cluster and the upgrade fails, and if you subsequently run tkg get cluster to see the list of deployed clusters and their versions, the cluster for which the upgrade failed shows the upgraded version of Kubernetes.

    Workaround: None

Kubernetes Issues
  • Tanzu Mission Control reports Tanzu Kubernetes Grid 1.1.3 clusters as unhealthy when they are actually healthy

    If you use Tanzu Kubernetes Grid 1.1.3 to deploy clusters with Kubernetes v1.17.9 and v1.18.6, and if you register these clusters with Tanzu Mission Control, Tanzu Mission Control reports that these clusters are unhealthy. This happens because these versions of Kubernetes introduced a change that affects the way that Tanzu Mission Control checks cluster health.

    This issue will be addressed in an update of Tanzu Mission Control.

check-circle-line exclamation-circle-line close-line
Scroll to top icon