Creating new variables in CredHub

You can use CredHub to create new variables and manage variables in the context of a larger deployment.

When a tile author defines a top-level variables section in the product template, VMware Tanzu Operations Manager passes the variables section to the product manifest.

You can define variables in the product template as follows:

variables:
  - name: EXAMPLE-CREDHUB-PASSWORD
    type: password

You can reference these variables in the manifest snippets in their tile metadata using triple parentheses syntax:

((( EXAMPLE-CREDHUB-PASSWORD )))

When you use triple parentheses, use VMware Tanzu Operations Manager to identify CredHub variables, while still supporting the BOSH double parentheses syntax. A variable that is referenced within triple parentheses is added with double parentheses in the generated manifest. After contacting CredHub, BOSH populates that variable value internally.

The benefit of this approach is that the VMware Tanzu Operations Manager YAML file does not contain sensitive credentials when the metadata manifest snippets have triple parentheses. The resulting manifest file contains variables within double parentheses, rather than unobscured credentials.

For example, a tile author adds credentials to a manifest snippet in the following format:

  key: ((( EXAMPLE-CREDHUB-PASSWORD )))
  key: prefix-((( ANOTHER-CREDHUB-PASSWORD )))-suffix

VMware Tanzu Operations Manager then evaluates the preceding example to generate the following section in the product manifest:

  (( EXAMPLE-CREDHUB-PASSWORD ))
  prefix-(( ANOTHER-CREDHUB-PASSWORD ))-suffix

How CredHub works in a deployment

CredHub is distributed as a BOSH release. As part of this installation, VMware Tanzu Operations Manager co-locates the CredHub release on the BOSH Director, including the CredHub job configurations, and the BOSH Director is configured to point to the CredHub API.

After CredHub is deployed and configured on the BOSH Director, any BOSH Director deployment can use CredHub variables in place of credential values. When you use variables, rather than values, it provides an extra layer of security when credentials are transmitted within your deployment.

Changing your deployment manifest to include CredHub variables

The BOSH Director interpolates credential values into manifests that use the ((variables)) syntax. When the BOSH Director encounters a variable using this syntax, it requests the credential value from CredHub. If the credential does not exist and the release or manifest contains generation properties, the credential value is generated automatically.

The following manifest excerpt includes references to two credentials, EXAMPLE-PASSWORD and EXAMPLE-TLS.

When this manifest is deployed, the BOSH Director retrieves the stored variables and replaces them with the credential values associated with each variable. The EXAMPLE-TLS variables include property accessors, because only the certificate and private_key components are interpolated.

name: demo-deploy

instance_groups:
  jobs:
  - name: demo
    release: demo
    properties:
      demo:
        password: ((EXAMPLE-PASSWORD))
        tls:
          certificate: ((EXAMPLE-TLS.certificate))
          private_key: ((EXAMPLE-TLS.private_key))

VMware Tanzu Operations Manager configures the BOSH Director to generate a credential if it does not exist. The manifest includes generation parameters that define how the credential must be generated. These generation parameters are defined in the variables section.

---
name: demo deploy

variables:
- name: EXAMPLE-PASSWORD
  type: password
- name: EXAMPLE-CA
  type: certificate
  options:
    is_ca: true
    common_name: 'Example Certificate Authority'
- name: EXAMPLE-TLS
  type: certificate
  options:
    ca: EXAMPLE-CA
    common_name: example.com

instance_groups:
  jobs:
  - name: demo
    release: demo
    properties:
      demo:
        password: (( EXAMPLE-PASSWORD ))
        tls:
          certificate: (( EXAMPLE-TLS.certificate ))
          private_key: (( EXAMPLE-TLS.private_key ))

Variable namespacing

Deployment manifests often use common variable names, for example, (( PASSWORD )). To avoid variable name collisions between deployments, the BOSH Director automatically stores variables with the BOSH Director name and deployment name.

For example, the variable (( EXAMPLE-PASSWORD )) is stored in CredHub as /BOSH-Director-name/deployment-name/example-password.

Other namespacing options

Use a BOSH link to share credentials across deployments. Alternatively, if you want to use an exact name, prefixing the variable with a forward slash (/) configures the BOSH Director to use the exact name you type.

Here is an example of a precisely typed variable:

((/EXAMPLE-PASSWORD))

Reference existing CAs in CredHub variables

This section describes how to correctly reference existing CAs stored in CredHub in your tile’s property configuration.

In VMware Tanzu Operations Manager v2.9 and later, you can perform a bulk rotation of all CAs and certificates in a foundation, which might include leaf certificates used by individual service tiles. VMware Tanzu Operations Manager invokes CredHub Maestro to perform this operation.

CredHub Maestro requires that any triple parentheses references to CAs that sign leaf certificates must return a concatenated version of the CA. The concatenated version, which includes the older and newer CA, ensures that jobs using leaf certificates do not lose trusted state during CA rotation. This translates to the shortest possible downtime of your tile’s services during certificate rotation.

When referencing a CA stored in CredHub, use the format LEAF-CERTIFICATE-NAME.ca to ensure that a concatenated version of the CA is returned. Do not reference the CA directly with the format CA-CERTIFICATE-NAME.certificate.

The following table presents examples of the correct and incorrect way to reference CAs and leaf certificates to support certificate rotation using CredHub Maestro.

Correct Format	Incorrect Format
templates: - name: bpm release: bpm - manifest: | ... .properties.routing_backends_client_cert_with_san.cert_pem )) private_key: (( .properties.routing_backends_client_cert_with_san.private_key_pem )) ca_certs: | (( .properties.routing_custom_ca_certificates.value )) (( $ops_manager.ca_certificate )) ((( /cf/some-diego-leaf-cert.ca ))) ((( /cf/some-diego-leaf-2-6.ca))) forwarded_client_cert: (( .properties.routing_tls_termination.selected_option.parsed_manifest(gorouter_forwarded_client_cert) )) variables: - name: /cf/diego-instance-identity-root-ca options: common_name: Diego Instance Identity Root CA duration: 1095 is_ca: true type: certificate - name: /cf/diego-instance-identity-root-ca-2-6 options: common_name: Diego Instance Identity Root CA duration: 1095 is_ca: true type: certificate - name: /cf/some-diego-leaf-cert.ca options: ca: /cf/diego-instance-identity-root-ca type: certificate - name: /cf/some-diego-leaf-2-6.ca options: ca: /cf/diego-instance-identity-root-ca-2-6 type: certificate	templates: - name: bpm release: bpm - manifest: | ... .properties.routing_backends_client_cert_with_san.cert_pem )) private_key: (( .properties.routing_backends_client_cert_with_san.private_key_pem )) ca_certs: | (( .properties.routing_custom_ca_certificates.value )) (( $ops_manager.ca_certificate )) ((( /cf/diego-instance-identity-root-ca.certificate ))) ((( /cf/diego-instance-identity-root-ca-2-6.certificate ))) forwarded_client_cert: (( .properties.routing_tls_termination.selected_option.parsed_manifest(gorouter_forwarded_client_cert) )) variables: - name: /cf/diego-instance-identity-root-ca options: common_name: Diego Instance Identity Root CA duration: 1095 is_ca: true type: certificate - name: /cf/diego-instance-identity-root-ca-2-6 options: common_name: Diego Instance Identity Root CA duration: 1095 is_ca: true type: certificate