Renewing Expired Certificates for IPsec for VMware Tanzu

This topic describes the basic process that you can use to renew any already expired certificates contained in the IPsec for VMware Tanzu manifest.

About Certificate Expiration

IPsec relies upon X.509 certificates to secure the communications between communicating peers.

Like all certificates, IPsec certificates have a finite lifetime and eventually expire. The certificates generated by the procedure provided in the installation instructions, Generate a Self-Signed Certificate have a default lifetime of one year. Regardless of their specific lifetime, all certificates must eventually be rotated, and so it is important for the operations team to plan accordingly and remember to rotate IPsec certificates before they actually expire.

Note Rotating the certificates while they are still valid ensures the maximum availability of the Cloud Foundry platform and avoids any unscheduled interruption in service.

Renew Expired IPsec Certificates

To renew expired IPsec certificates, do the following:

1. Retrieve the latest runtime config by running the following command:

   bosh -e BOSH-ENVIRONMENT runtime-config > PATH-TO-SAVE-THE-RUNTIME-CONFIG
   

2. Generate a new set of certificates. For development or test environments, you can use self-signed certificates. For information about self-signed certificates, see Generate a Self-Signed Certificate.

3. In the runtime config.yml file saved from step 1, update the optional field to true and update the certificate fields with new certificates. For more information about these fields, see the field descriptions under Create the IPsec Manifest.

   properties:
     ipsec:
       optional: true
       instance_certificate: |
         -----BEGIN CERTIFICATE-----
         EXAMPLEAhigAwIBAgIRAIvrBY2TttU/LeRhO+V1t0YwDQYJKoZIhvcNAQELBQAw
         ...
         -----END CERTIFICATE-----
       instance_private_key: |
         -----BEGIN EXAMPLE RSA PRIVATE KEY-----
         EXAMPLExRSAxPRIVATExKEYxDATAxEXAMPLExRSAxPRIVATExKEYxDATA
         ...
         -----END EXAMPLE RSA PRIVATE KEY-----
       ca_certificates:
         - |
           -----BEGIN CERTIFICATE-----
           EXAMPLEAvGgAwIBAgIBATANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDEwl0ZXN0
           ...
           -----END CERTIFICATE-----
   

4. Update the runtime config by running the following command:

   bosh -e BOSH-ENVIRONMENT update-runtime-config --name=ipsec PATH-TO-SAVE-THE-RUNTIME-CONFIG
   

5. Go to your Installation Dashboard in Tanzu Operations Manager.

6. If you are using Tanzu Operations Manager v2.3 or later, click Review Pending Changes. For more information about this Tanzu Operations Manager page, see Reviewing your pending product changes in Tanzu Operations Manager.
7. Click Apply Changes.
8. Set the optional field in the config.yml back to false.
9. Repeat steps 4 through 6 to update the runtime config again.
10. Click Apply Changes.

High-Availability (HA) Clusters

The IPsec add-on includes a setting named policy_optional that determines whether using IPsec for communication between the specified IP CIDRs is optional. If policy_optional is set to false, it is mandatory that communication be through IPsec among the specified IP CIDRs in the IPsec manifest. This can present a unique scenario while rotating the expired IPsec certificates for HA clusters, such as Tanzu Application Service’s deployment MySQL cluster. To work around this, we can temporarily make IPsec optional through the BOSH CLI and then rotate the certificates.

The workaround includes:

1. BOSH SSH into instance
2. monit stop ipsec
3. Update IPsec file to make policy_optional to true in the IPsec config file strongswan.conf
4. Run IPsec lifecycle scripts
5. Start IPsec once

This logic can be scripted.

Reference the following command for an example of how to make IPsec optional temporarily to reestablish communication between impacted VMs in a Tanzu Application Service deployment with deployment name “cf-1631c63accc0c5ca075c”.

Be sure to make any necessary adjustments to how BOSH CLI is used.

bosh -d INSERT_YOUR_DEPLOYMENT_NAME ssh -c "
sudo monit stop ipsec
sudo sed -i 's/policy_optional = no/policy_optional = yes/g' /var/vcap/jobs/ipsec/etc/strongswan.conf
sudo /var/vcap/jobs/ipsec/bin/post-stop
sudo /var/vcap/jobs/ipsec/bin/pre-start
sleep 10
sudo monit start ipsec"

After communication is reestablished, rotation of the IPsec certificates can take place. After this you must reverse the above change to set policy_optional to ‘no’.