Rotate Kubernetes Cluster Certificates

This topic describes how to rotate certificates used by VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) Kubernetes clusters.

For more information about TKGI Certificates:

• For conceptual information about certificates in TKGI, see TKGI Certificates.
• To rotate the certificates used by the TKGI control plane, see Rotating TKGI Control Plane Certificates.

Overview

When TKGI provisions a Kubernetes cluster, the system generates certificate authority (CA) certificates and leaf certificates that have values and expiration dates unique to that cluster.

The following table summarizes the TKGI-provisioned Kubernetes cluster certificates and how to rotate them.

Certificates	When Used	How to Rotate
kubo_master_ca_2021, kubo_ca_2018, etcd_ca_2018, and their leaf certificates	All clusters.	See Rotate Kubernetes Cluster Certificates below.
tls_nsx_t and tls_nsx_lb	NSX-T only. These certificates must be registered with NSX Manager.	See Rotate NSX-T Certificates for Kubernetes Clusters.

For more information about Kubernetes Cluster certificates in TKGI, see TKGI Certificates.

Warning: Never use the CredHub Maestro maestro regenerate ca/leaf –all command to rotate TKGI certificates.

Procedure

To rotate TKGI-provisioned Kubernetes cluster certificates, first determine which certificates are due to expire and then rotate them:

• List TLS Certificates
• Rotate TLS Certificates

List TLS Certificates

To list the TLS certificates used by TKGI-provisioned Kubernetes cluster, run the following command:

tkgi certificates CLUSTER-NAME -d DAYS

Where:

• CLUSTER-NAME is the name of the cluster.
• DAYS is the maximum number of days remaining until the certificate expires.

For example:

tkgi certificates tkgi-cluster-01 -d 10000

The sample output lists all TLS certificates that TKGI uses for the specified cluster.

NAME                                                                                            Type  Days Left  Valid until
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-nsx-lb                        Leaf  1803       2025-12-14T06:47:46Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-nsx-kube-proxy-2018           Leaf  1439       2024-12-15T06:47:41Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-ncp-2018                      Leaf  1439       2024-12-15T06:47:41Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-nsx-t                         Leaf  708        2022-12-15T06:47:40Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-kube-controller-manager-2018  Leaf  1439       2024-12-15T06:47:40Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-metrics-server-2018           Leaf  1439       2024-12-15T06:47:39Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-etcdctl-flanneld-2018-2       Leaf  1439       2024-12-15T06:47:39Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-etcdctl-root-2018-2           Leaf  1439       2024-12-15T06:47:38Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-etcdctl-2018-2                Leaf  1439       2024-12-15T06:47:37Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-etcd-2018-2                   Leaf  1439       2024-12-15T06:47:36Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-kubelet-client-2018           Leaf  1439       2024-12-15T06:47:36Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-kubelet-2018                  Leaf  1439       2024-12-15T06:47:35Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/etcd_ca_2018                      Root  1439       2024-12-15T06:47:35Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/tls-kubernetes-2018               Leaf  1439       2024-12-15T06:47:34Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/monitoring-metric-cert            Leaf  1439       2024-12-15T06:47:34Z
/p-bosh/service-instance_62e2a43a-dc2a-47a8-a361-3911589e60aa/kubo_ca_2018                      Root  1439       2024-12-15T06:47:34Z
 

Rotate TLS Certificates

The TKGI CLI supports rotating TLS certificates for the following scenarios:

• Rotate All Cluster Certificates
• Rotate All Cluster Certificates Except NSX-T
• Rotate NSX-T Certificates Only
• Rotate Custom CA

For more information about how to use TKGI CLI to rotate Kubernetes cluster TLS certificates, see Rotate TLS Certificates Using the TKGI CLI below.

Rotate All Cluster Certificates

To rotate all cluster certificates:

tkgi rotate-certificates CLUSTER-NAME --all

This command rotates all certificates except a custom CA kubo_master_ca_2021 (if implemented).

WARNING: Rotate cluster certificates only on a TKGI cluster that has been upgraded to the current TKGI version. For more information, see Tasks Supported Following a TKGI Control Plane Upgrade in About Tanzu Kubernetes Grid Integrated Edition Upgrades.

Rotate All Cluster Certificates Except NSX-T

To rotate all cluster certificates except the NSX-T certificates:

tkgi rotate-certificates CLUSTER-NAME --skip-nsx --all

This command rotates all certificates except tls-nsx-t and tls-nsx-lb.

WARNING: Rotate cluster certificates only on a TKGI cluster that has been upgraded to the current TKGI version. For more information, see Tasks Supported Following a TKGI Control Plane Upgrade in About Tanzu Kubernetes Grid Integrated Edition Upgrades.

Rotate NSX-T Certificates Only

To rotate only NSX certificates:

tkgi rotate-certificates CLUSTER-NAME --only-nsx

This command only rotates the NSX-T certificates tls-nsx-t and tls-nsx-lb.

For example:

tkgi rotate-certs tkgi-cluster-01 --only-nsx
 
You are about to rotate nsx related certificates for cluster tkgi-cluster-01. This operation requires bosh deployment, and will take a significant time. Are you sure you want to continue? (y/n):

For more information, see Rotate NSX-T Certificates for Kubernetes Clusters.

WARNING: Rotate cluster certificates only on a TKGI cluster that has been upgraded to the current TKGI version. For more information, see Tasks Supported Following a TKGI Control Plane Upgrade in About Tanzu Kubernetes Grid Integrated Edition Upgrades.

Rotate Custom CA

If you have implemented a custom CA for the kubo_master_ca_2021, rotation is handled by the update-cluster CLI command.

To rotate a custom kubo_master_ca_2021 CA:

1. If you are updating a cluster that uses a public cloud CSI driver, see Limitations on Using a Public Cloud CSI Driver in Release Notes for additional requirements.

2. Run the following command:

   tkgi update-cluster CLUSTER-NAME --config-file CONFIG-FILENAME
   

   Where:

   • CLUSTER-NAME is the name of the cluster.
   • CONFIG-FILENAME is the name of the configuration file.

   For complete usage, see Use a Custom CA for Kubernetes Clusters.

WARNING: Rotate cluster certificates only on a TKGI cluster that has been upgraded to the current TKGI version. For more information, see Tasks Supported Following a TKGI Control Plane Upgrade in About Tanzu Kubernetes Grid Integrated Edition Upgrades.

Rotate TLS Certificates Using the TKGI CLI

You can use the TKGI CLI to list and rotate the TLS certificates created for a Kubernetes cluster.

Usage:

tkgi rotate-certs | rotate-certificates CLUSTER-NAME [flags]

Flags:

      --all               Rotate all certs, not implemented yet, will be available in future releases.
  -h, --help              help for rotate-certs
      --json              Return the PKS-API output as json
      --non-interactive   Don't ask for user input
      --only-nsx          Rotate the tls-nsx-lb and tls-nsx-t certificates.
      --wait              Wait for the operation to finish

WARNING: Rotate cluster certificates only on a TKGI cluster that has been upgraded to the current TKGI version. For more information, see Tasks Supported Following a TKGI Control Plane Upgrade in About Tanzu Kubernetes Grid Integrated Edition Upgrades.