Secrets stores such as CredHub can be used to store secure properties that you don't want committed into a config file. Within your pipeline, the config file can then reference that secrets store value for runtime evaluation.
Platform Automation Toolkit contains two tasks to help with retrieving these credentials in the tasks that use them:
prepare-tasks-with-secrets
task can be used with any Concourse supported secrets store.credhub-interpolate
task can be used only with CredHub.The prepare-tasks-with-secrets
task takes a set of tasks and modifies them to include environment variables that reference the variables found in the config files provided. This allows use of the native Concourse secrets handling and provides support for any secret store Concourse supports.
The prepare-tasks-with-secrets
task replaces the credhub-interpolate task on Concourse versions 5.x+ and provides the following benefits:
The prepare-tasks-with-secrets
task can be used in two different ways:
credhub-interpolate
task Note When using prepare-tasks-with-secrets
, all secrets must exist in either a secrets store or a vars file found under VARS_PATHS
. If a vars from a config file can't be found in CredHub, it must be available in a YAML file found under VARS_PATHS
in prepare-tasks-with-secrets
. This will prevent these credentials from being added as environment variables to the task; this would result in Concourse being unable to find them in the secrets store.
To understand how prepare-tasks-with-secrets
modifies the Platform Automation Toolkit tasks, the following is an example of how a task will be changed:
Authenticate with your CredHub instance.
Generate a username and password.
credhub generate --name="/concourse/:team_name/:pipeline_name/vcenter_login" --type=user --username=some-user
Create a director configuration file that references the properties using the om
interpolation syntax.
properties-configuration:
iaas_configuration:
vcenter_host: ((vcenter_host))
vcenter_username: ((vcenter_login.username))
vcenter_password: ((vcenter_login.password))
(Optional) Create vars files with additional variables not stored in the secrets store.
This is recommended only for non-secret variables. It's more secure to store secrets in the secrets store. If using multiple foundations, there are some cases where a foundation-specific key might not be sensitive, but should be extracted to allow reuse of the config file between foundations. If using a single config file for multiple foundations, vars files may be used instead of storing the variables in a secrets store.
For example:
vcenter_host: vcenter.example.com
Configure your pipeline to use the prepare-tasks-with-secrets
task.
config
input is required and is a directory that contains your configuration file from (3).tasks
input is required and is the set of tasks that will be modified.vars
input and VARS_PATHS
param are only required if vars files are being used in subsequent tasks.output_mapping
section is required and is where the modified tasks will be.The declaration in a pipeline might look similar to this example:
- task: prepare-tasks-with-secrets
file: platform-automation-tasks/tasks/prepare-tasks-with-secrets.yml
image: platform-automation-image
input_mapping:
tasks: platform-automation-tasks
config: deployments
vars: deployments # required only if using vars
output_mapping:
tasks: platform-automation-tasks
params:
CONFIG_PATHS: ((foundation))/config
VARS_PATHS: ((foundation))/vars # required only if using vars
Note Unlike with credhub-interpolate, there is no concept of SKIP_MISSING
. As such, if there are credentials that will be filled in future jobs by vars files, those vars files must be provided in the vars
input and the VARS_PATHS
param.
This task will replace all of the tasks provided in the tasks
input with the modified tasks. The modified tasks include an extended params
section with the secret references detected from the config files.
Use the modified tasks in future jobs.
Here's an example of what prepare-tasks-with-secrets
is doing internally. Given an original task and the previously provided config/vars files:
# Original Platform Automation Toolkit Task
platform: linux
inputs:
- name: platform-automation-tasks
- name: config # contains the director configuration file
- name: env # contains the env file with target OpsMan Information
- name: vars # variable files to be made available
optional: true
- name: secrets
# secret files to be made available
# separate from vars, so they can be store securely
optional: true
- name: ops-files # operations files to custom configure the product
optional: true
params:
VARS_FILES:
# - Optional
# - Filepath to the Tanzu Operations Manager vars YAML file
# - The path is relative to root of the task build,
# so `vars` and `secrets` can be used.
OPS_FILES:
# - Optional
# - Filepath to the Tanzu Operations Manager operations YAML files
# - The path is relative to root of the task build
ENV_FILE: env.yml
# - Required
# - Filepath of the env config YAML
# - The path is relative to root of the `env` input
DIRECTOR_CONFIG_FILE: director.yml
# - Required
# - Filepath to the director configuration YAML file
# - The path is relative to the root of the `config` input
run:
path: platform-automation-tasks/tasks/configure-director.sh
The prepare-tasks-with-secrets
task modifies the original task to embed the variables found in director.yml
, in the params
section. Any variables found in the vars.yml
file will not be included in the modified task. The params
added will have a prefix of OM_VAR
, to avoid collisions. The task is a programmatically modified YAML file, so the output loses the comments and keys are sorted.
# prepare-job-with-secrets Generated Task
inputs:
- name: platform-automation-tasks
- name: config
- name: env
- name: vars
optional: true
- name: secrets
optional: true
- name: ops-files
optional: true
params:
DIRECTOR_CONFIG_FILE: director.yml
ENV_FILE: env.yml
OM_VAR_vcenter_password: ((vcenter_password))
OM_VARS_ENV: OM_VAR
OPS_FILES:
VARS_FILES:
platform: linux
run:
path: platform-automation-tasks/tasks/configure-director.sh
If you already have implemented the credhub-interpolate
task in your pipeline, this solution should be a drop-in replacement if you are not using vars files. You must be using Concourse 5.x or greater.
If you are using vars files, the vars
input and the VARS_PATHS
param will also need to be set on the prepare-tasks-with-secrets
task.
For example, if the existing credhub-interpolate
task looks like this:
---
platform: linux
inputs:
- name: platform-automation-tasks
- name: files
# contains YAML files with extension `.yml`.
# Each one of these files will have their values interpolated from credhub.
# For examples, run: `credhub interpolate --help`
# (minimum version >= 2.1.0 required)
outputs:
- name: interpolated-files
# Contains only YAML files found and interpolated by this task.
# Maintains the filestructure of the `files` input.
# all params are required to be filled out
params:
CREDHUB_CLIENT:
CREDHUB_SECRET:
CREDHUB_SERVER:
# - Required
# - Credentials to talk to credhub server
CREDHUB_CA_CERT:
# - Optional
# - This is only necessary if your Concourse worker
# is not already configured to trust the CA used for CredHub.
# - If more than one CA cert is required (ie the UAA),
# the CA certs can be concatenated together and separated by a newline.
# For example,
# CREDHUB_CA_CERT: |
# -----BEGIN CERTIFICATE-----
# ...credhub cert...
# -----END CERTIFICATE-----
# -----BEGIN CERTIFICATE-----
# ...UAA cert...
# -----END CERTIFICATE-----
PREFIX:
# - Required
# - Prefix flag used by credhub interpolate
INTERPOLATION_PATHS: '.'
# - Required
# - Path the contains the files to read from
# - This is a space separated list of directories
# the paths are all evaluated relative to files/
SKIP_MISSING: true
# Optional
# Change to false to have strict interpolation
# and fail if params are missing from vars
run:
path: platform-automation-tasks/tasks/credhub-interpolate.sh
In this task definition, you defined the prefix and CredHub authorization credentials. The new prepare-tasks-with-secrets
task uses Concourse's native integration with CredHub (and other credential managers). The task definition can be replaced with the following:
- task: prepare-tasks-with-secrets
file: platform-automation-tasks/tasks/prepare-tasks-with-secrets.yml
input_mapping:
tasks: platform-automation-tasks
config: configuration
vars: configuration # required only if using vars
output_mapping:
tasks: platform-automation-tasks
params:
CONFIG_PATHS: ((foundation))/config
VARS_PATHS: ((foundation))/vars # required only if using vars
Note If using vars files in subsequent tasks, the vars
input and the VARS_PATHS
param must be used to prevent interpolation errors in those subsequent tasks.
For the preceding, also note:
output_mapping
, which is required. This will replace all platform-automation-tasks
with the modified tasks. The modification now includes an extended params
, which now includes the secrets references detected from the config files.INTERPOLATION_PATHS
is now CONFIG_PATHS
. The concept of reading the references from the config files is still here, but no interpolation actually happens.PREFIX
is no longer defined or provided. Since the tasks are using Concourse's native credential management, the lookup path is predetermined. For example, /concourse/:team_name/:cred_name
or /concourse/:team_name/:pipeline_name/:cred_name
.The credhub-interpolate task can only be used with CredHub.
If you are using Concourse 5.x+, it is recommended that you use the prepare-tasks-with-secrets task instead.
An example workflow: Storing an SSH key
Authenticate with your CredHub instance.
Generate an SSH key: credhub generate --name="/concourse/:team_name/:pipeline_name/opsman_ssh_key" --type=ssh
Create an Tanzu Operations Manager configuration file that references the name of the property.
opsman-configuration:
azure:
ssh_public_key: ((opsman_ssh_key.public_key))
Configure your pipeline to use the credhub-interpolate task. It takes an input called files
, which should contain your configuration file from the earlier step.
The declaration in a pipeline might look like this:
jobs:
- name: example-job
plan:
- get: platform-automation-tasks
- get: platform-automation-image
- get: config
- task: credhub-interpolate
image: platform-automation-image
file: platform-automation-tasks/tasks/credhub-interpolate.yml
input_mapping:
files: config
params:
# depending on CredHub configuration
# ether CA cert or secret are required
CREDHUB_CA_CERT: ((credhub_ca_cert))
CREDHUB_SECRET: ((credhub_secret))
# all required
CREDHUB_CLIENT: ((credhub_client))
CREDHUB_SERVER: ((credhub_server))
PREFIX: /concourse/:team_name/:pipeline_name
SKIP_MISSING: true
This task will reach out to the deployed CredHub and fill in your entry references and return an output named interpolated-files
that can then be read as an input to any tasks that follow.
Our configuration will now look like this:
opsman-configuration:
azure:
ssh_public_key: ssh-rsa AAAAB3Nz...
Note If using this, you need to ensure that the Concourse worker can talk to CredHub. Depending on how you deployed CredHub and/or the worker, this may not be possible. Using credhub-interpolate
inverts control; now workers need to access CredHub. With prepare-tasks-with-secrets
and other uses of the Concourse native integration, the ATC retrieves secrets from CredHub and passes them to the worker.
There are three ways to include certificates in the YAML files that are used by Platform Automation Toolkit tasks.
Direct inclusion in YAML file:
# An incomplete base.yml response from om staged-config
product-name: cf
product-properties:
.uaa.service_provider_key_credentials:
value:
cert_pem: |
-----BEGIN CERTIFICATE-----
...<Some Cert>...
-----END CERTIFICATE-----
private_key_pem: |
-----BEGIN RSA PRIVATE KEY-----
...<Some Private Key>...
-----END RSA PRIVATE KEY-----
.properties.networking_poe_ssl_certs:
value:
-
certificate:
cert_pem: |
-----BEGIN CERTIFICATE-----
...<Some Cert>...
-----END CERTIFICATE-----
private_key_pem: |
-----BEGIN RSA PRIVATE KEY-----
...<Some Private Key>...
-----END RSA PRIVATE KEY-----
Secrets manager reference in YAML file:
# An incomplete base.yml
product-name: cf
product-properties:
.uaa.service_provider_key_credentials:
value:
cert_pem: ((uaa_service_provider_key_credentials.certificate))
private_key_pem: ((uaa_service_provider_key_credentials.private_key))
.properties.networking_poe_ssl_certs:
value:
-
certificate:
cert_pem: ((networking_poe_ssl_certs.certificate))
private_key_pem: ((networking_poe_ssl_certs.private_key))
This example assumes the use of CredHub.
CredHub supports a --type=certificate
credential type which allows you to store a certificate and private key pair under a single name. The cert and key can be stored temporarily in local files or can be passed directly on the command line.
An example of the file storage method:
credhub set --type=certificate \
--name=uaa_service_provider_key_credentials \
--certificate=./cert.pem \
--private=./private.key
Using vars files:
Vars files are a mix of the two previous methods. The cert/key is defined inline in the vars file.
#vars.yml
uaa_service_provider_key_credentials_cert_pem: |
-----BEGIN CERTIFICATE-----
...<Some Cert>...
-----END CERTIFICATE-----
uaa_service_provider_key_credentials_private_key: |
-----BEGIN RSA PRIVATE KEY-----
...<Some Private Key>...
-----END RSA PRIVATE KEY-----
networking_poe_ssl_certs_cert_pem: |
-----BEGIN CERTIFICATE-----
...<Some Cert>...
-----END CERTIFICATE-----
networking_poe_ssl_certs_private_key: |
-----BEGIN RSA PRIVATE KEY-----
...<Some Private Key>...
-----END RSA PRIVATE KEY-----
It is then referenced as a ((parameter))
in the base.yml
.
# An incomplete base.yml
product-name: cf
product-properties:
.uaa.service_provider_key_credentials:
value:
cert_pem: ((uaa_service_provider_key_credentials_cert_pem))
private_key_pem: ((uaa_service_provider_key_credentials_private_key))
.properties.networking_poe_ssl_certs:
value:
-
certificate:
cert_pem: ((networking_poe_ssl_certs_cert_pem))
private_key_pem: ((networking_poe_ssl_certs_private_key))
If you have multiple foundations, store relevant keys to each foundation in a different pipeline path, and Concourse reads the values in appropriately. When sharing the same base.yml
across foundations, it is recommended that you have a different pipeline for each foundation.
Vars files can be used for your secrets handling. They are not recommended, but are sometimes required based on your foundation setup.
Consider the following example (which only uses vars files and does not use a secrets store):
# base.yml
# An incomplete YAML response from om staged-config
product-name: cf
product-properties:
.cloud_controller.apps_domain:
value: ((cloud_controller_apps_domain))
.cloud_controller.encrypt_key:
value:
secret: ((cloud_controller_encrypt_key.secret))
.properties.security_acknowledgement:
value: X
.properties.cloud_controller_default_stack:
value: default
network-properties:
network:
name: DEPLOYMENT
other_availability_zones:
- name: AZ01
singleton_availability_zone:
name: AZ01
resource-config:
diego_cell:
instances: 5
instance_type:
id: automatic
uaa:
instances: 1
instance_type:
id: automatic
The first foundation has the following vars.yml
, optional for the configure-product
task.
# vars.yml
cloud_controller_encrypt_key.secret: super-secret-encryption-key
cloud_controller_apps_domain: cfapps.domain.com
The vars.yml
can then be passed to configure-product
with base.yml
as the config file. The configure-product
task will then substitute the ((cloud_controller_encrypt_key.secret))
and ((cloud_controller_apps_domain))
specified in vars.yml
and configure the product as normal.
An example of how this might look in a pipeline. (Resources are not listed in this example):
jobs:
- name: configure-product
plan:
- aggregate:
- get: platform-automation-image
params:
unpack: true
- get: platform-automation-tasks
params:
unpack: true
- get: configuration
- get: variable
- task: configure-product
image: platform-automation-image
file: platform-automation-tasks/tasks/configure-product.yml
input_mapping:
config: configuration
env: configuration
vars: variable
params:
CONFIG_FILE: base.yml
VARS_FILES: vars.yml
ENV_FILE: env.yml
If deploying more than one foundation, a unique vars.yml
should be used for each foundation.
Using prepare-tasks-with-secrets
and vars files: CredHub and vars files may be used together to interpolate variables into base.yml
. This use case is described in Using prepare-tasks-with-secrets.
Using credhub-interpolate
and vars files: CredHub and vars files may be used together to interpolate variables into base.yml
. Using the same example from earlier in this topic.
Using credhub-interpolate
and vars files together:
# base.yml
# An incomplete YAML response from om staged-config
product-name: cf
product-properties:
.cloud_controller.apps_domain:
value: ((cloud_controller_apps_domain))
.cloud_controller.encrypt_key:
value:
secret: ((cloud_controller_encrypt_key.secret))
.properties.security_acknowledgement:
value: X
.properties.cloud_controller_default_stack:
value: default
network-properties:
network:
name: DEPLOYMENT
other_availability_zones:
- name: AZ01
singleton_availability_zone:
name: AZ01
resource-config:
diego_cell:
instances: 5
instance_type:
id: automatic
uaa:
instances: 1
instance_type:
id: automatic
There is one parametrized variable that is secret and you might not want to have stored in a plain text vars file, ((cloud_controller_encrypt_key.secret))
, but ((cloud_controller_apps_domain))
is fine in a vars file.
To support a base.yml
with credentials from multiple sources (that is, CredHub and vars files), you must use SKIP_MISSING: true
in the credhub-interpolate
task. This is enabled by default by the credhub-interpolate
task.
The workflow is the same as CredHub, but when passing the interpolated base.yml
as a config into the next task, you add in a vars file to fill in the missing variables.
An example of how this might look in a pipeline (resources not listed), assuming:
((base.yml))
above((cloud_controller_encrypt_key.secret))
is stored in CredHub((cloud_controller_apps_domain))
is stored in director-vars.yml
jobs:
- name: example-credhub-interpolate
plan:
- get: platform-automation-tasks
- get: platform-automation-image
- get: config
- task: credhub-interpolate
image: platform-automation-image
file: platform-automation-tasks/tasks/credhub-interpolate.yml
input_mapping:
files: config
params:
# depending on CredHub configuration
# ether CredHub CA cert or CredHub secret are required
CREDHUB_CA_CERT: ((credhub_ca_cert))
CREDHUB_SECRET: ((credhub_secret))
# all required
CREDHUB_CLIENT: ((credhub_client))
CREDHUB_SERVER: ((credhub_server))
PREFIX: /concourse/:team_name/:pipeline_name
SKIP_MISSING: true
- name: example-configure-director
plan:
- get:
- task: configure-director
image: platform-automation-image
file: platform-automation-tasks/tasks/configure-director.yml
params:
VARS_FILES: vars/director-vars.yml
ENV_FILE: env/env.yml
DIRECTOR_CONFIG_FILE: config/director.yml
When using the credhub-interpolate
task with a CredHub in a single foundation or multi-foundation, it's best to avoid duplicating identical credentials (duplication makes credential rotation more difficult).
To have CredHub read in credentials from multiple paths (not relative to your PREFIX
), you must provide the absolute path to any credentials not in your relative path.
For example, using an alternative base.yml
:
# An incomplete YAML response from om staged-config
product-name: cf
product-properties:
.cloud_controller.apps_domain:
value: ((cloud_controller_apps_domain))
.cloud_controller.encrypt_key:
value:
secret: ((/alternate_prefix/cloud_controller_encrypt_key.secret))
.properties.security_acknowledgement:
value: X
.properties.cloud_controller_default_stack:
value: default
Suppose there is an example job
in which the prefix is defined as "foundation1." The parameterized values in the previous example are interpolated as follows:
((cloud_controller_apps_domain))
uses a relative path for CredHub. When running credhub-interpolate
, the task prepends the PREFIX
. This value is stored in CredHub as /foundation1/cloud_controller_apps_domain
.
((/alternate_prefix/cloud_controller_encrypt_key.secret))
(note the leading slash) uses an absolute path for CredHub. When running credhub-interpolate
, the task does not prepend the prefix. This value is stored in CredHub at its absolute path /alternate_prefix/cloud_controller_encrypt_key.secret
.
Any value with a leading /
slash will never use the PREFIX
to look up values in CredHub, so you can have multiple key lookups in a single interpolate task.