Use the Embedded Harbor Registry with Tanzu Kubernetes Clusters

You can use the embedded Harbor Registry to serve as the private container registry for images that you deploy to Tanzu Kubernetes clusters provisioned by the Tanzu Kubernetes Grid Service.

vSphere with Tanzu embeds a Harbor Registry instance that you can enable on the Supervisor Cluster and use to deploy container-based workloads to Tanzu Kubernetes clusters.

Once the embedded Harbor Registry is enabled on the Supervisor Cluster, the Tanzu Kubernetes Grid Service will install onto the Tanzu Kubernetes cluster nodes the root CA certificate for the registry instance. This certificate is installed on both new clusters and on existing clusters (by way of a reconciliation loop). From there you can run images on the cluster by specifying the private registry in the workload YAML.

Workflow

Use the following workflow to securely access the private registry from Tanzu Kubernetes cluster nodes and pull container images.

Step	Action	Instructions
0	Review the workflow for using the embedded Harbor Registry with Tanzu Kubernetes clusters.	See Using a Container Registry for vSphere with Tanzu Workloads.
1	Enable the embedded Harbor Registry on the Supervisor Cluster.	See Enable the Embedded Harbor Registry on the Supervisor Cluster.
2	Configure the kubeconfig for each cluster with the Registry Service Secret.	See the instructions below: Configure a Tanzu Kubernetes cluster with the image pull secret for the embedded Harbor Registry.
3	Configure the workload YAML to specify the private container registry.	See the instructions below: Configure a Tanzu Kubernetes cluster with the image pull secret for the embedded Harbor Registry.
4	To push images to the embedded Harbor Registry, configure a Docker client and install the vSphere Docker Credential Helper.	See Configure a Docker Client with the Embedded Harbor Registry Certificate and Push Images to the Embedded Harbor Registry.

Configure a Tanzu Kubernetes Cluster with the Image Pull Secret for the Embedded Harbor Registry

Configure your kubeconfig with the image pull secret to connect a Tanzu Kubernetes cluster to a private container registry, either the embedded Harbor Registry or an external private registry.

Connect to the Supervisor Cluster. See Connect to the Supervisor Cluster as a vCenter Single Sign-On User.
Switch context to the vSphere Namespace where the target Tanzu Kubernetes cluster is provisioned.
```
kubectl config use-context tkgs-cluster-ns
```

Get the image pull secret for the vSphere Namespace and store it in a file.

kubectl get secret -n <vsphere-namespace> <vsphere-namespace>-default-image-pull-secret -o yaml > <path>/image-pull-secret.yaml

For example:

kubectl get secret -n tkgs-cluster-ns tkgs-cluster-ns-default-image-pull-secret -o yaml > tanzu/image-pull-secret.yaml

Open image-pull-secret.yaml with a text editor. At a minimum make the required change and save the file when you are done.
```
apiVersion: v1
data:
  .dockerconfigjson: ewoJCQkJImF1dGhzJUV2s1ZVZwWVFuWmp...
kind: Secret
metadata:
  creationTimestamp: "2020-11-12T02:41:08Z"
  managedFields:
  - apiVersion: v1
    ...
  name: harbor-registry-secret   #OPTIONAL: Change if desired
  namespace: default           #REQUIRED: Enter the Kubernetes namespace
  ownerReferences:
  - apiVersion: registryagent.vmware.com/v1alpha1
    ...
  resourceVersion: "675868"
  selfLink: /api/v1/namespaces/tkgs-cluster-ns/secrets/tkgs-cluster-ns-default-image-pull-secret
  uid: 66606b41-7363-4b74-a3f2-4436f83f
type: kubernetes.io/dockerconfigjson
```
- REQUIRED: Change the value for namespace to match an appropriate Kubernetes namespace in the cluster, such as default.
  Note: To configure the image pull secret, specify a Kubernetes namespace. If the Tanzu Kubernetes cluster already exists, switch context to it and run kubectl get namespaces to list available Kubernetes namespaces. If necessary create the target namespace before proceeding. If the Tanzu Kubernetes cluster does not exist, you can use the default namespace.
- OPTIONAL: Change the value for name to something meaningful, such as harbor-registry-secret or private-registry-secret.

Create a kubeconfig file that can be used to access the Tanzu Kubernetes cluster.

kubectl get secret -n <vsphere-namespace> <cluster-name>-kubeconfig -o jsonpath='{.data.value}' | base64 -d > <path>/cluster-kubeconfig

Replace <vsphere-namespace> with the name of the vSphere Namespace where the target Tanzu Kubernetes cluster is provisioned. Replace <cluster-name> with the name of the Tanzu Kubernetes cluster.

kubectl get secret -n tkgs-cluster-ns tkgs-cluster-5-kubeconfig -o jsonpath='{.data.value}' | base64 -d > tanzu/cluster-kubeconfig

Create the Registry Service secret in the Tanzu Kubernetes cluster. Reference the image pull secret file that you saved and updated locally.
```
kubectl --kubeconfig=<path>/cluster-kubeconfig apply -f <path>/image-pull-secret.yaml
```
For example:
```
kubectl --kubeconfig=tanzu/cluster-kubeconfig apply -f tanzu/image-pull-secret.yaml
```
You should see that the Registry Service secret is successfully created.
```
secret/harbor-registry-secret created
```

Configure a Tanzu Kubernetes Workload to Pull Images from a Private Container Registry

To pull images from a private container registry for a Tanzu Kubernetes cluster workload, configure the workload YAML with the private registry details.

This procedure can be used to pull images from a private container registry, or the embedded Harbor Registry. In this example, we create a pod specification that will use an image stored in the embedded Harbor Registry and utilize the image pull secret previously configured.

Create an example pod spec with the details about the private registry.
```
apiVersion: v1
kind: Pod
metadata:
  name: <workload-name>
  namespace: <kubernetes-namespace>
spec:
  containers:
  - name: private-reg-container
    image: <Registry-IP-Address>/<vsphere-namespace>/<image-name>:<version>
  imagePullSecrets:
  - name: <registry-secret-name>
```
- Replace <workload-name> with the name of the pod workload.
- Replace <kubernetes-namespace> with the Kubernetes namespace in the cluster where the pod will be created. This must be the same Kubernetes namespace where the Registry Service image pull secret is stored in the Tanzu Kubernetes cluster (such as the default namespace).
- Replace <Registry-IP-Address> with the IP address for the embedded Harbor Registry instance running on the Supervisor Cluster.
- Replace <vsphere-namespace> with the vSphere Namespace where the target Tanzu Kubernetes is provisioned.
- Replace <image-name> with an image name of your choice.
- Replace <version> with an appropriate version of the image, such as "latest".
- Replace <registry-secret-name> with the name of the Registry Service image pull secret that you created previously.
Create a workload in the Tanzu Kubernetes cluster based on the pod specification you defined.
```
kubectl --kubeconfig=<path>/cluster-kubeconfig apply -f <pod.yaml>
```
The pod should be created from the image pulled from the registry.