Use a Custom CA for Kubernetes Clusters

This topic describes how to use a custom certificate authority (CA) to secure VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) provisioned Kubernetes clusters.

Custom CA Support

By default TKGI creates a new per-cluster CA (kubo_master_ca_2021) for each Kubernetes cluster. TKGI manages the lifecycle of the per-cluster CA and the certificates it signs. For more information, see Overview of Kubernetes Cluster Certificates.

For most use cases, the system-managed per-cluster CA is appropriate. If it is required, you can apply a custom CA to a Kubernetes cluster, either during creation or update.

Configuring a Kubernetes cluster with a custom CA is an advanced operation. If you use a custom CA, you are responsible for managing its lifecycle, including monitoring its expiry and rotating it while still valid.

To use custom CA functionality, existing clusters must be upgraded to support per-cluster CA.

Warning: Using a custom CA is an advanced operation. It is recommended that you contact VMware Support before proceeding.

Create Cluster with Custom CA

To apply a custom CA to a new cluster, specify the custom CA using the --config-file option with the create-cluster operation.

For example:

tkgi create-cluster <clustername> --external-hostname <clustername.example.com> --plan 'Plan 1' --config-file custom_ca.yaml

See custom CA requirements and formats.

Update Cluster with Custom CA

To apply a custom CA to an existing cluster, specify the custom CA using the --config-file option with the update-cluster operation.

For example:

tkgi update-cluster <clustername> --config-file custom_ca.yaml

See custom CA requirements and formats.

WARNING: Update the configuration file only on a TKGI cluster that has been upgraded to the current TKGI version. For more information, see Tasks Supported Following a TKGI Control Plane Upgrade in About Tanzu Kubernetes Grid Integrated Edition Upgrades.

Rotate Custom CA

You can rotate a custom CA by updating the cluster with the new CA.

Note: This procedure is designed to work if the custom CA you are replacing has not expired.

For example:

tkgi update-cluster <clustername> --config-file custom_ca_new.yaml

See custom CA requirements and formats.

Troubleshoot Custom CA

The log is located at /var/vcap/sys/log/pks-api.log.

The following command will trigger a Bosh deployment. You can track the progress using the Bosh CLI:

 bosh tasks
 bosh task <task-id> --debug

After rotation of the custom CA, check the cluster status and certificate expiration date.

Custom CA Requirements

The custom CA must include a certificate and private key in standard PEM format as follows:

{
"custom_ca": 
  {
    "certificate": "certificate in PEM format",
    "private_key": "private key in PEM format"
  }
}

The custom CA must satisfy following validation rules:

It must be a CA
It must be a root CA
It must not expire within 6 months
The private key and the certificate must match

Custom CA Formats

TKGI supports both YAML (*.yml or *.yaml) and JSON (*.json) configuration file format for supplying the custom CA.

Below is an example configuration file in YAML format for a custom CA (custom_ca.yaml):

# YAML for custom CA with certificate and private key
---
custom_ca:
  certificate: |
    -----BEGIN CERTIFICATE-----
    MIIDdTCCAl2gAwIBAgIUAyYwzsbEoRKrd+j2L74Noh3CDEEwDQYJKoZIhvcNAQEL
    ...
    ...
    BWlVhWvqBkTMTCNE83gGLPvRgTWqSqsd+Q==
    -----END CERTIFICATE-----
  private_key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEowIBAAKCAQEAti6NiJny8D+dfFZ2TXY4VWTyjyr/tFuVLr9XcwOhe9q/lQ6E
    ...
    vf3/DKcf9K/iHfdrfDDO/fVE/TO/4Gc1qmc2Arw1uWruXlqV2+6w
    -----END RSA PRIVATE KEY-----

Below is an example configuration file in JSON format for a custom CA (custom_ca.json):

{
   "custom_ca":{
      "certificate":"-----BEGIN CERTIFICATE-----\nMIIDdTCCAl2gAwIBAgIUAy...83TWqSqsd+Q==\n-----END CERTIFICATE-----\n",
      "private_key":"-----BEGIN RSA PRIVATE KEY-----\nMIEowIBAAKCAti6iJny...XlqV2+6w\n-----END RSA PRIVATE KEY-----\n"
   }
}