Installing Concourse with BOSH

Concourse for VMware Tanzu is available for installation as a BOSH release. If you haven't read the Prerequisites and Background Information page, please do so before continuing.

This guide describes a process for installing VMware Concourse as a BOSH release for CI/CD usage. If you're interested in setting up Platform Automation, navigate to the Installing VMware Concourse for Platform Automation instead.

Set up Certificates, Log in, and Alias your BOSH Environment

If you already have your CA certificate and have already logged in to and aliased your BOSH environment, you can skip this section.

Get a CA certificate for your BOSH Director.
1. If you created your BOSH Director manually, retrieve the credentials that were created during setup.
  
  If you are working with an Operations Manager-deployed BOSH Director, log in to Operations Manager and access the following endpoint in your Tanzu Operations Manager domain:
```
https://OPS-MANAGER-DOMAIN/download_root_ca_cert
```
  Where OPS-MANAGER-DOMAIN is the Operations Manager domain.
  
  This will create a file in your downloads folder called root_ca_certificate.
2. Copy and rename the root_ca_certificate file into the working directory. For example, on OSX:
```
mv ~/Downloads/root_ca_certificate ./ca-cert.crt
```
This certificate file can be used whenever a CLI command asks for a --ca-cert flag and value.
Ensure you are logged in to your BOSH environment with the appropriate BOSH Director credentials. For example, if you saved your CA certificate as ca-cert.crt, run the following command:
```
bosh -e BOSH-ENVIRONMENT-IP login --ca-cert=ca-cert.crt
```
Where BOSH-ENVIRONMENT-IP is your BOSH environment IP address.

If you set up a BOSH Director directly, the director credentials were returned as a file after the setup process finished.

If you have an Operations Manager-deployed BOSH Director, you can find the director credentials in the Operations Manager credentials tab or at the following endpoint in your Operations Manager domain:
```
http://OPS-MANAGER-DOMAIN/api/v0/deployed/director/credentials/director_credentials
```
Where OPS-MANAGER-DOMAIN is your Operations Manager domain.
Give your environment an alias by running the following command:
```
bosh -e BOSH-ENVIRONMENT-IP alias-env ALIAS --ca-cert=ca-cert.crt
```
Where:
- BOSH-ENVIRONMENT-IP is your BOSH environment IP address
- ALIAS is the alias you're creating BOSH environment
Using an alias for your BOSH environment substantially reduces the keystrokes needed for commands in future. You can use this alias whenever you target this environment by using the -e flag in a BOSH command.

Set up concourse-bosh-deployment directory on your local machine

The concourse-bosh-deployment repository has a sample BOSH manifest, versions.yml file, and a selection of deployment-modifying operations files. Using these sample files makes it much faster and easier to get started.

Clone the concourse-bosh-deployment repo by running the following snippet on the command line:
```
git clone https://github.com/concourse/concourse-bosh-deployment.git
```
Move to the concourse-bosh-deployment directory:
```
cd concourse-bosh-deployment
```
All the paths used in this tutorial are relative to this directory.
Checkout the release that corresponds to the version of Concourse you want to install. For example, if you're installing the latest release, Concourse v7.4.4:
```
git checkout v7.4.4
```
For a list of all Concourse releases, see concourse-bosh-deployment in GitHub.

When you check out a release (instead of a branch), Git notifies you: "You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout."

Download and Upload Concourse Release

If you haven't already, download the appropriate Concourse release from the Broadcom Support portal. For example, if you were to use the Concourse v7.4.0 release, you should end up with a file called concourse-bosh-release-7.4.0.tgz in your Downloads directory.
Use the bosh upload-release command to upload the Concourse tarball that you downloaded from the Broadcom Support portal.

For example, with the latest release, Concourse v7.4.0:
```
bosh -e BOSH-ENVIRONMENT upload-release ~/Downloads/concourse-bosh-release-7.4.0.tgz
```
Click here for more information about uploading releases.

BOSH and Concourse variables

Installing Concourse requires setting certain configuration variables that depend on infrastructure. In this section we'll use the bosh cloud-config command to look up our available options, and create a file to store the aforementioned configuration variables. We'll include this file in our final deployment.

Run the following command to fetch the cloud configuration details of your IaaS and put them in a cloud-config.yml file to reference later:

bosh \
-e BOSH-ENVIRONMENT-ALIAS \
cloud-config > cloud-config.yml

The following steps describe setting variables that are necessary for deployment in a yaml file. If you'd prefer to set them on the command line when you run bosh deploy, you could pass them in at that time with the --var KEY=VALUE flag and syntax. Click here to learn more about bosh deploy.

Create a file called variables.yml file to store your Concourse- and BOSH-related environment variables. You can do this using vim or your favorite editor:
```
vim variables.yml
```
Fill in the following variables:
```
---
deployment_name: DEPLOYMENT-NAME
azs:
- AVAILABILITY-ZONES
network_name: NETWORK-NAME
web_vm_type: VM-TYPE
worker_vm_type: VM-TYPE
db_persistent_disk_type: PERSISTENT-DISK-TYPE
db_vm_type: VM-TYPE
postgres_password: POSTGRES-PASSWORD
external_url: EXTERNAL-URL

web_ip: WEB-IP
local_user:
  username: USERNAME
  password: PASSWORD
external_host: EXTERNAL-HOST
encryption_key: DB-ENCRYPTION-KEY
```
Where:
- DEPLOYMENT-NAME is the name of your choice for your Concourse deployment
- AVAILABILITY-ZONES is an array of name that corresponds to azs in your cloud-config.yml file.
- NETWORK-NAME is the name of one of the networks in cloud-config.yml
- VM-TYPE is the name of one of the VM types in your cloud-config.yml file
- PERSISTENT-DISK-TYPE is the name of one of the disk types in cloud-config.yml
- POSTGRES-PASSWORD is the password of your choice for the PostgreSQL db used by Concourse
- EXTERNAL-URL is a url associated with the web_ip property (eg, http://WEB-IP:8080)
- WEB-IP is a non-reserved IP address for your deployment (eg, 1.1.1.1). Only required if ./cluster/operations/static-web.yml is used
- local_user.username your choice of username, used to log in to Concourse. Only required if ./cluster/operations/basic-auth.yml is used
- local_user.password is your choice of password, used to log in to Concourse.. Only required if ./cluster/operations/basic-auth.yml is used
- EXTERNAL-HOST is the domain to be used on the TLS certificate. Only required if ./cluster/operations/tls-vars.yml is used
- DB-ENCRYPTION-KEY is the key used to encrypt the Concourse database. Only required if ./cluster/operations/encryption.yml is used
Reserved IP ranges:
In some cases, you might need to modify the reserved IP ranges of your network on Operations Manager before choosing a web_ip for Concourse. For example, the reserved IP range is: 10.0.5.1-10.0.5.10, and you want to set web_ip as 10.0.5.8, the new reserved IP range to set on Operations Manager would be: 10.0.5.1-10.0.5.7, 10.0.5.9-10.0.5.10.

Concourse uses an external URL to facilitate communication between workers and ATCs.
When deploying for the first time, you might not know the final external URL. For example, you may later want to add a load balancer, or change your configuration as you go.
For now, choose any available URL to get started. You can always change this value later if you run into trouble or change your mind. To do so at any time, update the value in your variable files and redeploy.
Save and close the variables.yml file.

Set up a Deployment

To configure Concourse for VMware Tanzu, do the following:

Prepare the Concourse deployment manifest.

You can write your own manifest or modify a template with your unique configuration. The concourse-bosh-deployment repository that we cloned earlier has an example manifest template in the concourse.yml file located at ./cluster/concourse.yml.

For more information about Cluster Concourse deployment, see the README.md files at concourse-bosh-deployment in GitHub.
Prepare a cloud-config.yml file and upload it to your BOSH Director with bosh update-cloud-config.

Deploy Concourse with bosh deploy

To deploy your new Concourse, use the following information:

Your edited concourse manifest (eg, ./cluster/concourse.yml),
The versions file that comes with the concourse-bosh-deployment (eg, -l versions.yml),
Your environment variables (eg, -l variables.yml),
Any required operations files for your environment or Concourse setup.

The snippet below has a basic set of operations files, but you can add whatever options your unique deployment might require.
A --vars-store flag and a filename where BOSH can store your cluster credentials (eg, --vars-store cluster-creds.yml)

BOSH creates the cluster-creds.yml file for you if it does not already exist. If this is the case, you still need to specify a filename for BOSH to use. In the case that you already have an existing `cluster-creds.yml` file, specify it here so that BOSH does not re-create it.

Keep in mind that if you specify a filename that does not exist, and BOSH re-creates this file for you, it also re-creates things such as passwords or secrets that go inside the file.

When you are ready, move to your terminal and run the following bosh deploy commands:

bosh deploy \
-e BOSH-ENVIRONMENT-ALIAS \
-d CONCOURSE-DEPLOYMENT-NAME \
./cluster/concourse.yml \
-l versions.yml \
-l variables.yml \
-o ./cluster/operations/backup-atc.yml \
-o ./cluster/operations/basic-auth.yml \
-o ./cluster/operations/static-web.yml \
-o ./cluster/operations/privileged-https.yml \
-o ./cluster/operations/tls-vars.yml \
-o ./cluster/operations/tls.yml \
-o ./cluster/operations/encryption.yml \
--vars-store cluster-creds.yml

Make sure all your file paths are correct. For more information, see Deploying in the BOSH documentation.

Upon successfully running your deploy script, a success message is displayed, and Concourse has been installed.

Use the external URL that you specified in your variables.yml file to target your concourse with fly and view the Concourse web UI with command fly -t NEW-TARGET-NAME login -c http://WEB-IP:8080 -u USERNAME -p PASSWORD.

Troubleshooting

Different Concourse deployments require different environment variables and operations files. If you get an error, check the error message for clues about additional variables that need to be set. Check out the open-source Concourse documentation for additional information.