Install Tanzu Application Platform through Gitops with Secrets OPerationS (SOPS)

This topic tells you how to install Tanzu Application Platform (commonly known as TAP) through GitOps with secrets managed in a Git repository.

Caution
Tanzu Application Platform (GitOps) is currently in beta and is intended for evaluation and test purposes only. Do not use in a production environment.

Tanzu GitOps Reference Implementation (RI) does not support changing the secrets management strategy for a cluster.

Prerequisites

Before installing Tanzu Application Platform, you need:

SOPS CLI to view and edit SOPS encrypted files. To install the SOPS CLI, see SOPS documentation in GitHub.
Age CLI to create an encryption key used to encrypt and decrypt sensitive data. To install the Age CLI, see age documentation in GitHub.
Completed the Prerequisites.
Installed Tanzu CLI with any required plug-ins.
Installed Cluster Essentials for Tanzu.

Relocate images to a registry

Before installation, you must relocate the Tanzu Application Platform images from tanzu.packages.broadcom.com to your own container image registry.

The supported registries are Harbor, Azure Container Registry, Google Container Registry, and Quay.io. See the following documentation to learn how to set up your container image registry:

To relocate images from tanzu.packages.broadcom.com to your registry:

Retrieve your Broadcom registry API token:
1. Sign in to the Broadcom Support Portal.
2. Go to Tanzu Application Platform (TAP) and expand the VMware Tanzu Application Platform dropdown.
3. Click the Token Download icon next to the Tanzu Application Platform version you want to download.
4. Follow the instructions in the dialog box. Save the token as a variable named MY_BROADCOM_SUPPORT_ACCESS_TOKEN. For example:
```
export MY_BROADCOM_SUPPORT_ACCESS_TOKEN=API-TOKEN
```
  Where API-TOKEN is your token from the Broadcom Support Portal.

Set up environment variables for installation use by running:

# Set tanzu.packages.broadcom.com as the source registry to copy the Tanzu Application Platform packages from.
export IMGPKG_REGISTRY_HOSTNAME_0=tanzu.packages.broadcom.com
export IMGPKG_REGISTRY_USERNAME_0=MY-BROADCOM-SUPPORT-USERNAME
export IMGPKG_REGISTRY_PASSWORD_0=${MY_BROADCOM_SUPPORT_ACCESS_TOKEN}

# The user’s registry for copying the Tanzu Application Platform package to.
export IMGPKG_REGISTRY_HOSTNAME_1=MY-REGISTRY
export IMGPKG_REGISTRY_USERNAME_1=MY-REGISTRY-USER
export IMGPKG_REGISTRY_PASSWORD_1=MY-REGISTRY-PASSWORD
# These environment variables starting with IMGPKG_* are used by the imgpkg command only.

# The registry from which the Tanzu Application Platform package is retrieved.
export INSTALL_REGISTRY_USERNAME="${IMGPKG_REGISTRY_USERNAME_1}"
export INSTALL_REGISTRY_PASSWORD="${IMGPKG_REGISTRY_PASSWORD_1}"
export INSTALL_REGISTRY_HOSTNAME="${IMGPKG_REGISTRY_HOSTNAME_1}"
export TAP_VERSION=VERSION-NUMBER
export INSTALL_REPO=TARGET-REPOSITORY

Where:

MY-BROADCOM-SUPPORT-USERNAME is the user with access to the images in tanzu.packages.broadcom.com.
MY-REGISTRY is your own container registry.
MY-REGISTRY-USER is the user with write access to MY-REGISTRY.
MY-REGISTRY-PASSWORD is the password for MY-REGISTRY-USER.
VERSION-NUMBER is your Tanzu Application Platform version. For example, 1.8.7.
TARGET-REPOSITORY is your target repository. This is a folder or repository on MY-REGISTRY that serves as the location for the Tanzu Application Platform installation files.

VMware recommends using a JSON key file to authenticate with Google Container Registry. In this case, the value of INSTALL_REGISTRY_USERNAME is _json_key and the value of INSTALL_REGISTRY_PASSWORD is the content of the JSON key file. For more information about how to generate the JSON key file, see Google Container Registry documentation.

Install the Carvel tool imgpkg CLI.

To query for the available versions of Tanzu Application Platform on tanzu.packages.broadcom.com, run:
```
imgpkg tag list -i tanzu.packages.broadcom.com/tanzu-application-platform/tap-packages | sort -V
```

Relocate the images with the imgpkg CLI by running:

imgpkg copy -b tanzu.packages.broadcom.com/tanzu-application-platform/tap-packages:${TAP_VERSION} --to-repo ${INSTALL_REGISTRY_HOSTNAME}/${INSTALL_REPO}/tap-packages

(Optional) Install Tanzu Application Platform in an air-gapped environment

Complete the following steps if you install Tanzu Application Platform in an air-gapped environment:

Retrieve the Tanzu Build Service version by running:

kubectl get package -n tap-install | grep buildservice

Relocate the Tanzu Build Service images to your registry by running:

imgpkg copy -b tanzu.packages.broadcom.com/tanzu-application-platform/full-tbs-deps-package-repo:TBS-VERSION --to-repo ${INSTALL_REGISTRY_HOSTNAME}/${INSTALL_REPO}/full-tbs-deps-package-repo

Where TBS-VERSION is the version of Tanzu Build Service you retrieved.

Host a grype database in the air-gapped environment. For more information, see Use vulnerability scanning in offline and air-gapped environments.

Create a new Git repository

Follow these steps to create a new Git repository and set up the authentication:

In a hosted Git service, for example, GitHub or GitLab, create a new repository.

This version of Tanzu GitOps RI supports authenticating to a hosted Git repository by using SSH and Basic Authentication.

Initialize a new Git repository:

mkdir -p $HOME/tap-gitops
cd $HOME/tap-gitops

git init
git remote add origin [email protected]:my-organization/tap-gitops.git

Set up the authentication method.

SSH

Create a read-only deploy key for this new repository (recommended) or SSH key for an account with read access to this repository. The private portion of this key is called GIT_SSH_PRIVATE_KEY.

Basic Authentication

Have a user name with read access to the Git repository and password or personal access token for the same user.

Important
Only use one of ssh or Basic Authentication, not both.

Download and unpack Tanzu GitOps Reference Implementation (RI)

Download Tanzu GitOps Reference Implementation for Tanzu Application Platform v1.8.7 from the Broadcom Support Portal.
Unpack the downloaded TGZ file into the $HOME/tap-gitops directory by running:
```
tar xvf tanzu-gitops-ri-*.tgz -C $HOME/tap-gitops
```

Commit the initial state:

cd $HOME/tap-gitops

git add . && git commit -m "Initialize Tanzu GitOps RI"
git push -u origin

Create cluster configuration

Seed configuration for a cluster using SOPS:
```
cd $HOME/tap-gitops

./setup-repo.sh CLUSTER-NAME sops
```
Where:
- CLUSTER-NAME the name of your cluster.
- sops selects the Secrets OPerationS-based secrets management variant.
Example:
```
cd $HOME/tap-gitops

./setup-repo.sh full-tap-cluster sops
Created cluster configuration in ./clusters/full-tap-cluster.
...
```
This script creates the directory clusters/full-tap-cluster/ and copies in the configuration required to sync this Git repository with the cluster and installing Tanzu Application Platform.

Commit and push:

git add . && git commit -m "Add full-tap-cluster"
git push

Configure Tanzu Application Platform

Tanzu Sync Reference Implementation (RI) splits the values configuration of Tanzu Application Platform into two categories:

Sensitive TAP values, for example, credentials, encryptions keys and so on.
Non-sensitive TAP values, for example, packages to exclude, namespace configuration and so on.

The following sections describe how to create these values files.

Prepare the sensitive Tanzu Application Platform values

Follow these steps to prepare the sensitive Tanzu Application Platform values:

Generate Age public or secrets keys:

Note
Skip this step if you already have an Age key to encrypt or decrypt secrets.

mkdir -p $HOME/tmp-enc
chmod 700 $HOME/tmp-enc
cd $HOME/tmp-enc

age-keygen -o key.txt

cat key.txt
# created: 2023-02-08T10:55:35-07:00
# public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
AGE-SECRET-KEY-my-secret-key

Create a plain YAML file tap-sensitive-values.yaml outside of your Git repository that contains a placeholder for the sensitive portion of the Tanzu Application Platform values:
```
---
tap_install:
  sensitive_values:
```
Encrypt tap-sensitive-values.yaml with Age using SOPS:
```
export SOPS_AGE_RECIPIENTS=$(cat key.txt | grep "# public key: " | sed 's/# public key: //')
sops --encrypt tap-sensitive-values.yaml > tap-sensitive-values.sops.yaml
```
Where:
- grep is used to find the line containing the public key portion of the generated secret.
- sed is used to extract the public key from the line found by grep.
(Optional) Verify the encrypted file can be decrypted:
```
export SOPS_AGE_KEY_FILE=key.txt
sops --decrypt tap-sensitive-values.sops.yaml
```
(Optional) Verify the encrypted file can be edited directly by using SOPS:
```
sops tap-sensitive-values.sops.yaml
```

Move the sensitive Tanzu Application Platform values into the cluster config:

mv tap-sensitive-values.sops.yaml GIT-REPO-ROOT/clusters/CLUSTER-NAME/cluster-config/values/

Example:

mv tap-sensitive-values.sops.yaml $HOME/tap-gitops/clusters/full-tap-cluster/cluster-config/values/

(Optional) Retain the Age identity key file in a safe and secure place such as a password manager, and purge the scratch space:
```
mv key.txt SAFE-LOCATION/
export SOPS_AGE_KEY_FILE=SAFE-LOCATION/key.txt
rm -rf $HOME/tmp-enc
```

Prepare the non-sensitive Tanzu Application Platform values

Create a plain YAML file GIT-REPO-ROOT/clusters/CLUSTER-NAME/cluster-config/values/tap-non-sensitive-values.yaml by using the Full Profile sample as a guide:

Example:

---
tap_install:
  values:
    ceip_policy_disclosed: true
    excluded_packages:
    - policy.apps.tanzu.vmware.com
...

To install Tanzu Application Platform in an offline environment, you must configure Tanzu Build Service and Grype to work in an air-gapped environment:

---
tap_install:
  values:
    ...
    buildservice:
      exclude_dependencies: true
    grype:
      db:
        dbUpdateUrl: INTERNAL-VULN-DB-URL

Where INTERNAL-VULN-DB-URL is the URL that points to the internal file server.

For more information, see Components and installation profiles.

Update the sensitive Tanzu Application Platform values

After filling in the non-sensitive values, follow these steps to extract the sensitive values into tap-sensitive-values.sops.yaml that you prepared earlier:

Open an editor through SOPS to edit the encrypted sensitive values file:

sops GIT-REPO-ROOT/clusters/CLUSTER-NAME/cluster-config/values/tap-sensitive-values.sops.yaml

Example:

sops $HOME/tap-gitops/clusters/full-tap-cluster/cluster-config/values/tap-sensitive-values.sops.yaml

Add the sensitive values:

Example of the container registry credentials using basic authentication:

---
tap_install:
 sensitive_values:
   shared:
     image_registry:
       project_path: "example.com/my-project/tap"
       username: "my_username"
       password: "my_password"

Example of the container registry credentials using Google Container Registry:

---
tap_install:
 sensitive_values:
  shared:
    image_registry:
       project_path: "gcr.io/my-project/tap"
       username: "_json_key"
       password: |
         {
           "type": "service_account",
           "project_id": "my-project",
           "private_key_id": "my-private-key-id",
           "private_key": "-----BEGIN PRIVATE KEY-----\n..........\n-----END PRIVATE KEY-----\n",
           ...
         }

Prepare the sensitive Tanzu Sync values

Follow these steps to prepare the sensitive Tanzu Sync values:

Create a plain YAML file tanzu-sync-values.yaml outside of your Git repository that contains a placeholder for the sensitive portion of Tanzu Sync values:
```
---
secrets:
    sops:
```
Encrypt tanzu-sync-values.yaml with Age using SOPS:
```
export SOPS_AGE_RECIPIENTS=$(cat key.txt | grep "# public key: " | sed 's/# public key: //')
sops --encrypt tanzu-sync-values.yaml > tanzu-sync-values.sops.yaml
```
Where:
- grep is used to find the line containing the public key portion of the generated secret.
- sed is used to extract the public key from the line found by grep.

Move the encrypted sensitive Tanzu Application Platform values into the Tanzu Sync config:

mv tanzu-sync-values.sops.yaml GIT-REPO-ROOT/clusters/CLUSTER-NAME/tanzu-sync/app/sensitive-values/tanzu-sync-values.sops.yaml

Example:

mv tanzu-sync-values.sops.yaml $HOME/tap-gitops/clusters/full-tap-cluster/tanzu-sync/app/sensitive-values/tanzu-sync-values.sops.yaml

Update the sensitive Tanzu Sync values

Follow these steps to populate tap-sensitive-values.sops.yaml with credentials:

Open an editor and use SOPS to edit the encrypted sensitive values file:

sops GIT-REPO-ROOT/clusters/CLUSTER-NAME/tanzu-sync/app/sensitive-values/tanzu-sync-values.sops.yaml

Example:

sops $HOME/tap-gitops/clusters/full-tap-cluster/tanzu-sync/app/sensitive-values/tanzu-sync-values.sops.yaml

Add the sensitive values:

Configuration example for the Git SSH authentication:
```
---
secrets:
    sops:
        age_key: |
            AGE-KEY
        registry:
            hostname: MY-REGISTRY
            username: MY-REGISTRY-USER
            password: MY-REGISTRY-PASSWORD
        git:
            ssh:
                private_key: |
                    PRIVATE-KEY
                known_hosts: |
                    HOST-LIST
```
Configuration example for the Git basic authentication:
```
---
secrets:
    sops:
        age_key: |
            AGE-KEY
        registry:
            hostname: MY-REGISTRY
            username: MY-REGISTRY-USER
            password: MY-REGISTRY-PASSWORD
        git:
            basic_auth:
                username: MY-GIT-USERNAME
                password: MY-GIT-PASSWORD
```
Where:
- AGE-KEY is the contents of the Age key generated earlier.
- MY-REGISTRY is your container image registry.
- MY-REGISTRY-USER is the user with read access to MY-REGISTRY.
- MY-REGISTRY-PASSWORD is the password for MY-REGISTRY-USER.
- PRIVATE-KEY is the contents of an SSH private key file with read access to your Git repository. Only applies when you use ssh.
- HOST-LIST is the list of known hosts for Git host service. Only applies when you use ssh.
- MY-GIT-USERNAME is the user with read access to your Git repository. Only applies when you use basic authentication.
- MY-GIT-PASSWORD is the password for MY-GIT-USERNAME. Only applies when you use basic authentication.
You can find the schema for Tanzu Sync credentials in GIT-REPO-ROOT/clusters/CLUSTER-NAME/tanzu-sync/app/config/.tanzu-managed/schema--sops.yaml

Generate Tanzu Application Platform installation and Tanzu Sync configuration

Follow these steps to generate the Tanzu Application Platform installation and Tanzu Sync configuration:

Set up environment variables by running:
```
export TAP_PKGR_REPO=TAP-PACKAGE-OCI-REPOSITORY
export SOPS_AGE_KEY=AGE-KEY
```
Where:
- TAP-PACKAGE-OCI-REPOSITORY is the fully-qualified path to the OCI repository hosting the Tanzu Application Platform images. If the images are relocated as described in Relocate images to a registry, this value is ${INSTALL_REGISTRY_HOSTNAME}/${INSTALL_REPO}/tap-packages.
- AGE-KEY is the contents of the Age key generated earlier.
Example of the Git repo hosted on GitHub:
```
export TAP_PKGR_REPO=tanzu.packages.broadcom.com/tanzu-application-platform/tap-packages
export SOPS_AGE_KEY=$(cat $HOME/tmp-enc/key.txt)
```

Generate the Tanzu Application Platform install and the Tanzu Sync configuration files by using the provided script:

cd GIT-REPO-ROOT/clusters/CLUSTER-NAME

./tanzu-sync/scripts/configure.sh

Example:

cd $HOME/tap-gitops/clusters/full-tap-cluster

./tanzu-sync/scripts/configure.sh

(Optional) Update Tanzu Application Platform to use the latest patch:

Update the Tanzu Application Platform version in GIT-REPO-ROOT/clusters/CLUSTER-NAME/cluster-config/values/tap-install-values.yaml:
```
tap_install:
    ...
    version:
        package_repo_bundle_tag: "1.8.7" # Populate these values with the latest patch version.
        package_version: "1.8.7"
```
Where:
- package_repo_bundle_tag is the version of Tanzu Application Platform you want to upgrade to.
- package_version is the version of Tanzu Application Platform you want to upgrade to. This version must match package_repo_bundle_tag.
Note
Tanzu GitOps RI does not provide a separate artifact for each patch version within a minor line. For example, Tanzu Application Platform v1.6.x only contains the v1.6.1 GitOps artifact.

Commit the generated configured to Git repository:

git add cluster-config/ tanzu-sync/
git commit -m "Configure install of TAP 1.6.1"
git push

Deploy Tanzu Sync

Follow these steps to deploy Tanzu Sync:

Install the Carvel tools kapp and ytt onto your $PATH:

sudo cp $HOME/tanzu-cluster-essentials/kapp /usr/local/bin/kapp
sudo cp $HOME/tanzu-cluster-essentials/ytt /usr/local/bin/ytt

Set the Kubernetes cluster context.
1. List the existing contexts:
```
kubectl config get-contexts
```
2. Set the context to the cluster that you want to deploy:
```
kubectl config use-context CONTEXT-NAME
```
  Where CONTEXT-NAME can be retrieved from the outputs of the previous step.
Deploy the Tanzu Sync component:
```
cd GIT-REPO-ROOT/clusters/CLUSTER-NAME

./tanzu-sync/scripts/deploy.sh
```
Example:
```
cd $HOME/tap-gitops/clusters/full-tap-cluster

./tanzu-sync/scripts/deploy.sh
```
Note

Depending on the profile and components included, it may take 5-10 minutes for Tanzu Application Platform to install. During this time, kapp waits for the deployment of Tanzu Sync to reconcile successfully. This is normal.