This topic specifies the procedure to follow to check expiration dates and rotate certificates when using VMware Tanzu GemFire for Tanzu Application Service.
To rotate the Services TLS CA and its leaf certificates, for service instances that are not connected by WAN gateways, use one of the following procedures:
If you are running Ops Manager 2.10, follow the procedure in Rotate the Services TLS CA and Its Leaf Certificates in Advanced Certificate Rotation with CredHub Maestro in the VMware Tanzu Ops Manager 2.10 documentation.
Warning: For apps that connect to the Tanzu GemFire cluster using TLS, at each step in the procedure that rebinds and restages apps, recreate the truststore following the directions in Create a Truststore) in Accessing a Service Instanceprior to rebinding. Spring Boot Data Geode (SBDG) apps that set Tanzu GemFire properties ssl-use-default-context=true
and ssl-endpoint-identification-enabled=false
do not need to recreate a truststore.
If you are running Ops Manager 2.9, follow the procedure in Rotate the Services TLS CA and Its Leaf Certificates in Advanced Certificate Rotation with CredHub Maestro in the VMware Tanzu Ops Manager 2.9 documentation.
Warning: For apps that connect to the Tanzu GemFire cluster using TLS, at each step in the procedure that rebinds and restages apps, recreate the truststore following the directions in Create a Truststore) in Accessing a Service Instanceprior to rebinding. Spring Boot Data Geode (SBDG) apps that set Tanzu GemFire properties ssl-use-default-context=true
and ssl-endpoint-identification-enabled=false
do not need to recreate a truststore.
Follow this procedure for rotating the certificates of foundations, when service instances connected via WAN gateways reside on distinct foundations.
The procedure assumes two foundations have service instances, one called foundation A and the other called foundation B. The procedure may be followed by installations that have more than two foundations. To expand the procedure to cover more foundations, in each place where foundation A and foundation B are identified, also apply the directions for a third foundation C, a fourth foundation D, and so on.
The procedure is broken into parts. Do the parts in the order listed. The parts are:
Follow these steps twice to obtain a new TLS CA certificate, once to generate a new TLS CA certificate for foundation A, and once to generate a new TLS CA certificate for foundation B.
Check if CredHub has a new temporary certificate from a previous rotation attempt. Run:
credhub get -n /services/new_ca
Note: This procedure assumes that /services/new_ca
is the CredHub location used when generating the temporary certificate. If you used a different location, specify that location instead.
If an older temporary certificate already exists, delete that certificate by running:
credhub delete -n /services/new_ca
Do one of the following:
If you use self-signed certificates: Generate a new self-signed certificate with a new name or path in CredHub:
credhub generate \
--name="/services/new_ca" \
--type="certificate" \
--no-overwrite \
--is-ca \
--duration=1825 \
--common-name="opsmgr-services-tls-ca"
Where: - name
is the CredHub path or name of the new certificate. You can use /services/new_ca
or substitute a different name as long as you use the value consistently in the rest of the procedure. - common-name
is the common name of the generated certificate. You can use opsmgr-services-tls-ca
or substitute a different common name as long as you use the value consistently in the rest of the procedure. - duration
is set to 1825
, which is a recommended value of five years. If you do not specify a duration, the default value is 365
(one year).
If you use an intermediate certificate signed by a root CA in CredHub: Generate a new certificate signed by the CredHub root CA by running:
credhub generate \
--name="/services/new_ca" \
--type="certificate" \
--no-overwrite \
--is-ca \
--duration=1825 \
--common-name="opsmgr-services-tls-ca" \
--ca=/PATH-TO-ROOT-CA
Where: - name
is the CredHub path or name of the new certificate. You can use /services/new_ca
or substitute a different name as long as you use the value consistently in the rest of the procedure. - common-name
is the common name of the generated certificate. You can use opsmgr-services-tls-ca
or substitute a different common name as long as you use the value consistently in the rest of the procedure. - duration
is set to 1825
, which is a recommended value of five years. If you do not specify a duration, the default value is 365
(one year). - ca
is the path to the root CA for the new certificate. Replace PATH-TO-ROOT-CA
with the appropriate path for the root CA.
Each of foundation A and foundation B has a current certificate and the newly generated one from the previous step, totalling four certificates across two foundations.
On foundation A, log in to CredHub.
On foundation A, get the current Services TLS CA certificate by capturing the certificate in the output of:
credhub get --name=/services/tls_ca -k ca
On foundation A, get the new Services TLS CA certificate either from a pre-existing file, or from your new CredHub location by capturing the certificate in the output of:
credhub get --name=/services/new_ca -k ca
On foundation B, log in to CredHub.
On foundation B, get the current Services TLS CA certificate by capturing the certificate in the output of:
credhub get --name=/services/tls_ca -k ca
On foundation B, get the new Services TLS CA certificate either from a pre-existing file, or from your new CredHub location by capturing the certificate in the output of:
credhub get --name=/services/new_ca -k ca
This part adds all the certificates (current and new) to the tiles. For a distributed system with two foundations, the four certificates are:
/services/tls_ca
)/services/tls_ca
)/services/new_ca
)The new TLS CA certificate for foundation B (also called /services/new_ca
)
On foundation A, paste all certificates (ordering of the certificates does not matter) into the BOSH Director > Security > Trusted Certificates field. Click Save.
On foundation A, paste all certificates (ordering of the certificates does not matter) into two fields of the VMware Tanzu Application Service tile: Networking > Certificate Authorities trusted by the Gorouter and Networking > Certificate Authorities trusted by the HAProxy. Click Save.
On foundation A, paste all certificates (ordering of the certificates does not matter) into two fields of the Isolation Segment tile, if the environment has this optional tile: Networking > Certificate Authorities trusted by the Gorouter and Networking > Certificate Authorities trusted by the HAProxy. Click Save.
On foundation B, paste all certificates (ordering of the certificates does not matter) into the BOSH Director > Security > Trusted Certificates field. Click Save.
On foundation B, paste all certificates (ordering of the certificates does not matter) into two fields of the VMware Tanzu Application Service tile: Networking > Certificate Authorities trusted by the Gorouter and Networking > Certificate Authorities trusted by the HAProxy. Click Save.
On foundation B, paste all certificates (ordering of the certificates does not matter) into two fields of the Isolation Segment tile, if the environment has this optional tile: Networking > Certificate Authorities trusted by the Gorouter and Networking > Certificate Authorities trusted by the HAProxy. Click Save.
Warning: This procedure involves redeploying all of the VMs in your Tanzu Operations Manager deployment to apply a CA certificate. The operation can take a long time to complete.
You may apply the changes to both foundation A and foundation B at the same time.
Follow these steps twice, once to apply the changes to foundation A, and once to apply the changes to foundation B.
Identify which on-demand service tiles are using the Services TLS CA certificate. Run:
credhub curl -p /api/v1/certificates?name=%2Fservices%2Ftls_ca
From the returned list, identify which on-demand service tiles use TLS in your deployment, such as MySQL for VMware Tanzu, GemFire for Tanzu Application Service, Redis for VMware Tanzu, and RabbitMQ for VMware Tanzu [VMs].
Note: The name of the upgrade all service instances errand may differ slightly between services.
Set the new Services TLS CA certificate in CredHub. Do this on foundation A with foundation A’s new TLS CA certificate, and do this on foundation B with foundation B’s new TLS CA certificate.
Do one of the following:
If you have an existing certificate: Obtain the CA certificate and private key file corresponding to the certificate that you got in the Get New TLS CA Certificates part of the procedure. Then, run:
credhub set \
--name="/services/tls_ca" \
--type="certificate" \
--certificate=PEM-PATH/root.pem \
--private=CERT-KEY
Where: - certificate
is the location of the root.pem
file for the certificate. Replace PEM-PATH
with the path of your certificate’s root.pem
file. - private
is the private key for the new certificate. Replace CERT-KEY
with the value of the private key for your certificate.
If you created a new self-signed or intermediary certificate: Set the /services/new_ca
that you generated in the Get New TLS CA Certificates part of the procedure as the Services TLS CA:
credhub get -n /services/new_ca -k ca > new_ca.ca
credhub get -n /services/new_ca -k certificate > new_ca.certificate
credhub get -n /services/new_ca -k private_key > new_ca.private_key
credhub set -n /services/tls_ca \
--type=certificate \
--root=new_ca.ca \
--certificate=new_ca.certificate \
--private=new_ca.private_key
Warning: This procedure involves redeploying all of the VMs in your Tanzu Operations Manager deployment to apply a CA certificate. The operation can take a long time to complete.
You may apply the changes to both foundation A and foundation B at the same time.
In this step, you apply changes to ensure that all on-demand service instances generate new leaf certificates from the new Services TLS CA.
Follow these steps twice, once to apply the changes to foundation A, and once to apply the changes to foundation B.
Note: The name of the upgrade all service instances errand may differ slightly between services.
After your apps have reconnected to service instances with the certificates generated by the new CA, both of the old CA certificates may be removed. There will be one old CA certificate for foundation A and one old certificate for foundation B.
Follow these steps twice, once for foundation A, and once for foundation B.
Delete the two old CA certificates from the BOSH Director tile Security > Trusted Certificates field. You will remove one old CA certificate for foundation A and one old certificate for foundation B. Click Save.
Delete the two old CA certificates from the two fields of the VMware Tanzu Application Service tile: Networking > Certificate Authorities trusted by the Gorouter and Networking > Certificate Authorities trusted by the HAProxy. You will remove one old CA certificate for foundation A and one old certificate for foundation B. Click Save.
Delete the two old CA certificates from the two fields of the Isolation Segment tile, if the environment has this optional tile: Networking > Certificate Authorities trusted by the Gorouter and Networking > Certificate Authorities trusted by the HAProxy. You will remove one old CA certificate for foundation A and one old certificate for foundation B. Click Save.
As a security best practice, you should remove outdated certificates as soon as possible from your deployment. You can schedule this step to a convenient time, because for most deployments you will not lose any deployment functionality if you do not perform this step immediately.
For some deployments, you may encounter an error if you create a service instance in a deployment that contains an expired Services TLS CA certificate. For more information, see Cloud Controller fails to create service instance when there is a expired cert installed on the system in the Tanzu Support Knowledge Base.
Warning: This procedure involves redeploying all of the VMs in your Tanzu Operations Manager deployment to apply a CA certificate. The operation can take a long time to complete.
You may apply the changes to both foundation A and foundation B at the same time.
Follow these steps twice, once to apply the changes to foundation A, and once to apply the changes to foundation B.
Note: The name of the upgrade all service instances errand may differ slightly between services.