Synchronize VMware Tanzu Application Catalog Helm Charts in Air-Gapped environments

This section provides step-by-step instructions on how to use the charts-syncer tool to synchronize the VMware Tanzu Application Catalog (Tanzu Application Catalog) repository with your private Harbor repository using the charts-syncer command line tool. It also provides you instructions for deploying the charts-syncer tool in Kubernetes for unattended synchronization.

An air-gapped environment ensures that all the applications, networks, and resources in the environment are physically isolated from external inputs to prevent potential security risks.

In this context, if you need to operate with artifacts that are not within your network, you have to mirror them to the air-gapped environment, so that your teams are able to consume them under secure conditions.

To consume Tanzu Application Catalog content in an air-gapped enterprise environment, you can use a Harbor private repository installed in a private Kubernetes cluster and synchronize it with the Tanzu Application Catalog repository. This ensures that enterprise teams always have access to the most updated collection of Helm charts and container images without compromising the security of the air-gapped environment.

The best option to synchronize a Tanzu Application Catalog Helm Chart repository with a Harbor private repository in an air-gapped environment is to install the charts-syncer tool on the same Kubernetes cluster where Harbor is running. This tool is open source and therefore easily auditable. It is completely reliable and secure, since it is executed only within the air-gapped environment.

Note
Some Tanzu Application Catalog Helm charts containers have been verified for use in air-gapped environments. See How can I check if a Helm chart is appropriately configured to meet specific security requirements?

Understand the charts-syncer tool

Charts-syncer is a tool that helps you to relocate the Helm Chart repositories and associated container images from a source location to a target location. It also helps you keep them synchronized.

This guide focuses on the scenario in which the source and target Helm Chart/container registries are accessible from a common location, from where you can run the charts-syncer.

Understanding Charts-Syncer Tool

As part of the relocation, charts-syncer performs the following process for each Helm Chart that needs to be relocated:

Copies the Helm Chart associated container images to the target container registry.
Updates the Helm Chart container image references to point to the new location of the images.
Pushes the modified Helm Chart to the target Helm Chart repository or OCI registry.

What to do first

Ensure:

You have access to the Tanzu Application Catalog environment. If you do not have access, contact your VMware sales representative.
You have administrator access to a pre-existing installation of Harbor, configured in line with existing enterprise requirements. If you don’t already have Harbor, you can install it using Bitnami’s Harbor Helm chart. For more information, see Harbor website. Alternatively, having any compliant OCI registry should suffice that can be used to host the relocated Helm Chart and container images.

To Synchronize Tanzu Application Catalog Helm Charts in Air-Gapped environments, follow the steps as explained in the sections below:

Install the charts-syncer tool.
Configure and run the charts-syncer tool locally.
Deploy in Kubernetes.

Install the charts-syncer tool

Ensure that you have the latest version of the charts-syncer CLI. You can download the latest version from https://github.com/bitnami-labs/charts-syncer/releases.

Configure and run the charts-syncer tool locally

The following sections provide step-by-step instructions on how to configure and install the charts-syncer tool locally:

Create a configuration file

The next step is configuring the tool to sync both the source and the target repositories. This information is specified via a yaml file similar to the one below.

  source:
      repo:
        kind: HELM
        url: https://MY_REGISTRY/
      
  target:
    repo:
      kind: OCI
      url: https://HARBOR_DOMAIN/HARBOR_PROJECT/charts
    containerRegistry: HARBOR_DOMAIN
    containerRepository: HARBOR_DOMAIN/containers

  # Whether to also push the associated container images
  relocateContainerImages: true

  # While syncing Tanzu Application Catalog repositories we want to skip the "common" chart since it's a special case
  skipCharts:
      - common

Note
https://MY_REGISTRY/ is a placeholder. Replace it with the URL of the private registry where your catalog is located.

You should provide the following configuration information in the yaml file:

source: Refers to the repository from where the Helm Charts will be fetched. In this example, source is the Tanzu Application Catalog demo repository.
target.repo: This is the repository to which Tanzu Application Catalog pushes the charts. In this example, the charts are being pushed to an OCI repository backed by your Harbor instance. Note that traditional Helm repositories are supported as well.
containerRegistry and containerRepository: Indicates where to push the associated container images. This location will also be updated in the values.yaml file for the parent Helm Chart and subcharts.
relocateContainerImages: Instructs charts-syncer to not only relocate the Helm Charts but also the associated container images. This feature is compatible with Tanzu Application Catalog Helm Chart repositories. For more information, see Sync Helm Charts and Container Images.
HARBOR_DOMAIN and HARBOR_PROJECT: Replace the HARBOR_DOMAIN and HARBOR_PROJECT placeholders with your Harbor installation domain and the name of the project where you want to push the charts.

Note
Targeting the traditional Helm repositories provided by CharMuseum is also supported. This requires updating the tar. Update the target.repo section appropriately if you want to target the traditional ChartMuseum Helm repositories.


# config.yaml
…
target:
  repo:
    kind: CHARTMUSEUM
    url: https://CHARTMUSEUM_URL
…

Run charts-syncer tool

After updating the configuration file, enter the following command to run the charts-syncer tool:


$ charts-syncer sync -c config.yaml --latest-version-only

If authentication is required by either the source or the target repositories, a set of credentials should be provided via environment variables.

In the current example, only the target registry needs authentication.

Command invocation


$ TARGET_REPO_AUTH_USERNAME=my_target_repo_username \
  TARGET_REPO_AUTH_PASSWORD=my_target_repo_password \
  TARGET_CONTAINERS_AUTH_USERNAME=my_target_repo_username \
  TARGET_CONTAINERS_AUTH_PASSWORD=my_target_repo_password \
  charts-syncer sync -c config.yaml --latest-version-only -v3

TARGET_REPO_AUTH_USERNAME and TARGET_REPO_AUTH_PASSWORD are the credentials for the target repository to which the Helm Charts will be pushed.
TARGET_CONTAINERS_AUTH_USERNAME and TARGET_CONTAINERS_AUTH_PASSWORD are the credentials for the target container registry to which the associated container images will be pushed.

Note
In the current example, since both the Helm Chart and the container images are being pushed to the same Harbor instance, the credentials are the same. This may differ if you configure the charts-syncer tool to push charts and container images to different locations.

After the sync process is completed, you should be able to explore the relocated Helm Charts and container images through the Harbor UI.

Explore Helm Charts and Containers

As part of the relocation process, the charts-syncer tool updates the Helm Charts values.yaml files to point to the new location of the container images.

Updated YAML File

The relocated Helm Charts are now ready to be consumed by any Helm compliant client such as command line interfaces like Helm CLI proper, user interfaces like Kubeapps, or native Kubernetes controllers like flux or kapp-controller.

Deploy in Kubernetes

A native way of having two Helm Chart repositories synced is to run charts-syncer periodically using a Kubernetes CronJob.

This section explains how to specifically translate the fully-functional local environment set up from Step 2 into a Kubernetes deployment as explained in the Deploying charts-syncer in Kubernetes guide.

What to do first

Ensure:

You have successfully run charts-syncer locally and completed the previous steps in this guide.
You have a Kubernetes cluster and kubectl installed in your machine. See Getting Started with Kubernetes guide for an easy way to get started.

Procedure

Retrieve the charts-syncer repository which contains a set of Kubernetes deployment templates.
```
$ git clone https://github.com/bitnami-labs/charts-syncer.git
$ cd charts-syncer
```
Update the config.yaml file that will be used to run the tool in Kubernetes. To do this, copy the config.yaml file that is already working locally into deployment/config/config.yaml. For more information, refer to Step 1 in the Deploying charts-syncer in Kubernetes guide.
```
$ cp config.yaml deployment/config/config.yaml
```

Execute the kubectl, apply -k providing the same environment variables used during the local execution in Run charts-syncer tool.

$ TARGET_REPO_AUTH_USERNAME=my_target_repo_username \
TARGET_REPO_AUTH_PASSWORD=my_target_repo_password \
TARGET_CONTAINERS_AUTH_USERNAME=my_target_repo_username \
TARGET_CONTAINERS_AUTH_PASSWORD=my_target_repo_password \
kubectl apply -k ./deployment

This command configures and deploys *charts-syncer* in Kubernetes. For inspecting and troubleshooting the deployment, refer to [Step 4 - Try and debug an initial sync](https://github.com/bitnami-labs/charts-syncer/blob/HEAD/docs/kubernetes-deployment.md#step-4---try-and-debug-an-initial-sync) in the *Deploying charts-syncer in Kubernetes guide*.

## Appendix: Relocation on Disconnected Target Environment

In some situations, you may want to sync two Helm chart repositories without direct connectivity between them. The charts-syncer tool supports this scenario via intermediate Chart Bundles. For more information, see Air gap scenario.