Update Existing Clusters with Update CA Tool

After updating airgap server certificate, some features of cluster associating with this airgap collapse as those require pulling images or packages from the airgap server. This guide provides scripts to update existing cluster with new self-signed certificate or private root CA certificate to recover those features.

This tool is only applicable on TCA CP appliance. TCA cloud-native is not supported.

Prerequisites

Airgap server is updated with new self-signed certificate or private root CA signed certificate.

Note:

If airgap server is updated with public signed certificate, updating existing cluster or following the steps in this guide is not required.

Procedure

1. Download update CA tool tarball here and upload it to TCA CP appliance.

   Transfer the tarball to TCA CP appliance.

   # scp update-ca-v2.1.tar.gz admin@<tcacpip>:~
   # scp ca.crt admin@<tcacpip>:~
   # ssh admin@<tcacpip>
   [admin@tcacp ~]$ su
   Password:
   [root@tcacp /home/admin] ls
   update-ca-v2.1.tar.gz
   [root@tcacp /home/admin] tar vxfz update-ca-v2.1.tar.gz
   [root@tcacp /home/admin] ls
   update-ca update-ca-v2.1.tar.gz
   [root@tcacp /home/admin] cd update-ca
   [root@tcacp /home/admin/update-ca] ls
   ansible update_ca.py
   [root@tcacp /home/admin/update-ca] ./update_ca.py -h

2. Upload CA certificate of airgap server to TCA CP appliance.

   Save the CA certificate under /home/admin/, with file name ca.crt.

3. Run update-ca.py update-all command to show the components that are required to be updated. The clusters might reflect as out of date in the log.

   # update_ca.py verify-all --fqdn <airgap server fqdn> --cafile <new CA file path>

   # [root@tcacp /home/admin/update-ca]./update_ca.py verify-all --fqdn airgap-repo-server-2.ipv6.eng.vmware.com --cafile /home/admin/ca.crt

   All the clusters managed by this TCA CP will be traversed by the tool. The output of tool will show whether the corresponding components are required to be updated. Following output is an example:

   update_ca[INFO]: airgap repo: airgap-repo-server-2.ipv6.eng.vmware.com is valid
   update_ca[INFO]: ########## verifying tkgcontexts ##########
   update_ca[ERROR]: tkgcontext[5123887a-da9f-4312-8292-4d595c098a57]: out of date
   update_ca[ERROR]: tkgcontext[48fb570d-2226-4902-a893-3df2dbff97fd]: out of date
   update_ca[INFO]: ########## verifying management clusters ##########
   update_ca[INFO]: # verifying management cluster[mc210], id[a1d9dcc8-d43f-4278-909e-0a22019a21db]
   update_ca[ERROR]: configmap kapp-controller-config/tkg-system: out of date
   update_ca[ERROR]: configmap tkr-controller-config/tkr-system: out of date
   update_ca[ERROR]: kubecontrolplane cr[mc210-master-control-plane/tkg-system]: out of date
   update_ca[ERROR]: kubeadmconfigtemplate cr[mc210-np1/tkg-system]: out of date
   update_ca[ERROR]: node[172.16.69.14]: out of date, SSL certificate problem
   update_ca[ERROR]: node[172.16.68.231]: out of date, SSL certificate problem
   update_ca[INFO]: ########## verifying v1 workload clusters ##########
   update_ca[INFO]: # verifying workload cluster[wc210v1], id[57a25f87-4718-4597-85f2-20926005de78]
   update_ca[INFO]: configmap kapp-controller-config/tkg-system: up to date
   update_ca[ERROR]: kubecontrolplane cr[wc210v1-master-control-plane/wc210v1]: out of date
   update_ca[ERROR]: kubeadmconfigtemplate cr[wc210v1-np1/wc210v1]: out of date
   update_ca[INFO]: query cluster wc210v1 node ips
   update_ca[ERROR]: node[172.16.68.224]: out of date, SSL certificate problem
   update_ca[ERROR]: node[172.16.69.30]: out of date, SSL certificate problem
   update_ca[INFO]: ########## verifying minikube ##########
   update_ca[ERROR]: tcakubenetescluster cr[mc210/mc210]: out of date
   update_ca[INFO]: ########## verifying v2 workload clusters ##########
   update_ca[INFO]: # verifying v2 workload cluster[wc210v2]
   update_ca[ERROR]: tcakubenetescluster cr[wc210v2/wc210v2]: out of date
   update_ca[ERROR]: kubecontrolplane cr[wc210v2-cp-control-plane/wc210v2]: out of date
   update_ca[ERROR]: kubeadmconfigtemplate cr[wc210v2-np1/wc210v2]: out of date
   update_ca[INFO]: configmap kapp-controller-config/tkg-system: up to date
   update_ca[ERROR]: node[172.16.70.103]: out of date, SSL certificate problem

4. Run update_ca.py update-all command.

   # update_ca.py update-all --fqdn <airgap server fqdn> --cafile <new CA file path> --vcpass <vc password>

   Enable debug log for verbose outputs.

   # update_ca.py --loglevel debug update-all --fqdn <airgap server fqdn> --cafile <new CA file path> --vcpass <vc password>

   [root@tcacp /home/admin/update-ca] ./update_ca.py update-all --fqdn airgap-repo-server.example.com --cafile /home/admin/ca.crt --vcpass password

   This command will update all the components relevant with existing clusters associating with the specified airgap server.

   Attention:

   During the update, cluster control plane nodes will be redeployed which may lead to temporary access issue to the cluster, in case of single cluster control plane node. The script might fail in some step because of it.

   The update-ca.py commands is re-entrant, that could be run multiple times if any error shows.

5. Verify again with update_ca.py verify-all command as in Step 3. Return to Step 4 in case of any error message in the console.