Harbor is an open-source container registry. You can deploy the TKG Extension for Harbor Registry as a private registry store for the container images you want to deploy to Tanzu Kubernetes clusters.

Harbor Extension Version Dependencies

Adhere to the following minimum verision requirements for installing the TKG Extension for Harbor Registry in a Tanzu Kubernetes cluster provisioned by the Tanzu Kubernetes Grid Service.
Component Minimum Version
vCenter Server

7.0.2.00400

vSphere Namespace

0.0.10-18245956

Supervisor Cluster

v1.20.2+vmware.1-vsc0.0.10-18245956

Tanzu Kubernetes release

v1.20.7+vmware.1-tkg.1.7fb9067

Harbor Extension Prerequisites

Adhere to the following prerequisites before deploying the TKG Extension v1.3.1 for Harbor Registry.

Harbor Extension Additional Requirements

The TKG Extension v1.3.1 for Harbor Registry has additional requirements pre- and post-installation.
  • The Harbor extension requires a default PVC storage class. See Review Persistent Storage Requirements for TKG Extensions.
  • The Harbor extension requires HTTP/S ingress. Specifically, Harbor services are exposed via an Envoy service in the Contour Extension. As a prerequisite, deploy the Contour Extension. See Deploy and Manage the TKG Extension for Contour Ingress.
    • If you are using NSX-T networking for the Supervisor Cluster, create an Envoy service of type LoadBalancer.
    • If you are using vSphere vDS networking for the Supervisor Cluster, create an Envoy service of type LoadBalancer or of type NodePort, depending on your environment and requirements.
  • The Harbor extension requires DNS. After installation the Harbor extension you need to configure DNS.
    • For testing and verification purposes, add the Harbor and Notary FQDNs to your local /etc/hosts file. The instructions below describe how to do this.
    • In production Harbor requires a DNS Zone on either a local DNS Server, such as BIND, or on a public cloud, such as AWS Route53, Azure DNS, or Google CloudDNS. Once you have set up DNS, to automatically register the Harbor FQDNs with a DNS Server, install the External DNS extension. See Deploy and Manage the TKG Extension for External DNS Service Discovery.

Deploy the Harbor Extension

The TKG Extension for Harbor Registry installs several containers on the cluster. For more information, see https://goharbor.io/.
Container Resource Type Replicas Description
harbor-core Deployment 1 Management and configuration server for Envoy
harbor-database Pod 1 Postgres database
harbor-jobservice Deployment 1 Harbor job service
harbor-notary-server Deployment 1 Harbor notary service
harbor-notary-signer Deployment 1 Harbor notary
harbor-portal Deployment 1 Harbor web interface
harbor-redis Pod 1 Harbor redis instance
harbor-registry Deployment 2 Harbor container registry instance
harbor-trivy Pod 1 Harbor image vulnerability scanner
To install the Harbor Registry using the TKG Extension, complete the following steps.
  1. Verify that you have completed each of the extension prerequisites. See Harbor Extension Prerequisites and Harbor Extension Additional Requirements.
  2. Change directory to the Harbor extension.
    cd /tkg-extensions-v1.3.1+vmware.1/extensions/registry/harbor
  3. Create the tanzu-system-registry namespace and Harbor service account and roles.
    kubectl apply -f namespace-role.yaml
  4. Create a Harbor data values file.
    cp harbor-data-values.yaml.example harbor-data-values.yaml
  5. Specify the mandatory passwords and secrets in harbor-data-values.yaml.
    Harbor Registry requires several passwords and secrets listed and described in the table.
    Password or Secret Description
    harborAdminPassword The initial password for Harbor administrator.
    secretKey The secret key used for encryption. Must be a string of 16 characters.
    database.password The initial password for the Postgres database.
    core.secret Secret is used when core server communicates with other component.
    core.xsrfKey The XSRF key. Must be a string of 32 characters.
    jobservice.secret Secret is used when job service communicates with other component.
    registry.secret Secret is used to secure the upload state from client and registry storage backend.
    To automatically generate random passwords and secrets and populate the harbor-data-values.yaml file, run the following command:
    bash generate-passwords.sh harbor-data-values.yaml
    On success you should see the following message:
    Successfully generated random passwords and secrets in harbor-data-values.yaml

    Open the harbor-data-values.yaml file and verify the mandatory passwords and secrets.

  6. Specify other Harbor configuration values in harbor-data-values.yaml, if necessary. Commonly updated values may include the following:
    Configuration Field Description
    hostname

    The default Harbor hostname is core.harbor.domain.

    If necessary, change this value to match your requirements.

    port.https

    The default is 443.

    If you are using NSX-T networking for the Supervisor Cluster, and therefore an Envoy ingress service of type LoadBalancer, leave this setting as the default 443.

    If you are using vDS networking for the Supervisor Cluster, and therefore an Envoy ingress service of type NodePort, set this value to the match the Envoy node port.

    clair.enabled

    Clair image scanner is deprecated in favor of Trivy. Both are enabled in the configuration file.

    Disable Clair by setting its value to false.

    persistence.persistentVolumeClaim. <component>.accessMode

    There are several instances of this setting.

    The default is ReadWriteOnce.

    ReadWriteMany is scheduled to be supported in an upcoming release.

    imageChartStorage.type

    The default is filesystem.

    Change if necessary and configure the storage you are using.

    proxy If desired configure a proxy for Harbor. If a proxy is configured, the default noProxy values are required.
  7. Create a secret with the data values.
    kubectl create secret generic harbor-data-values --from-file=values.yaml=harbor-data-values.yaml -n tanzu-system-registry
    The secret/harbor-data-values is created in the tanzu-system-registry namespace. Verify this by running the following command:
    kubectl get secrets -n tanzu-system-registry
  8. Deploy the Harbor extension.
    kubectl apply -f harbor-extension.yaml

    On success you should see app.kappctrl.k14s.io/harbor created.

  9. Check the status of the Harbor application.
    kubectl get app harbor -n tanzu-system-registry
    On success the status changes from Reconciling to Reconcile succeeded.
    NAME     DESCRIPTION   SINCE-DEPLOY   AGE
    harbor   Reconciling   96s            98s
    NAME     DESCRIPTION           SINCE-DEPLOY   AGE
    harbor   Reconcile succeeded   39s            2m29s
    If the status is Reconcile failed, see Troubleshoot Harbor Registry Deployment.
  10. View detailed information on the Harbor extension.
    kubectl get app harbor -n tanzu-system-registry -o yaml
  11. View the status of the Harbor Deployment objects.
    kubectl get deployments -n tanzu-system-registry

    On success you should see the following Deployments:

    NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
    harbor-core            1/1     1            1           5m16s
    harbor-jobservice      1/1     1            1           5m16s
    harbor-notary-server   1/1     1            1           5m16s
    harbor-notary-signer   1/1     1            1           5m16s
    harbor-portal          1/1     1            1           5m16s
    harbor-registry        1/1     1            1           5m16s
  12. View the status of the Harbor pods:
    kubectl get pods -n tanzu-system-registry
    NAME                                    READY   STATUS    RESTARTS   AGE
    harbor-core-9cbf4b79d-gxvgx             1/1     Running   0          7m11s
    harbor-database-0                       1/1     Running   0          7m11s
    harbor-jobservice-6b656ccb95-lm47d      1/1     Running   0          7m11s
    harbor-notary-server-8494c684db-gm7jf   1/1     Running   0          7m11s
    harbor-notary-signer-6f96b549d4-dzcnm   1/1     Running   0          7m11s
    harbor-portal-5b8f4ddbd-qdnp2           1/1     Running   0          7m11s
    harbor-redis-0                          1/1     Running   0          7m11s
    harbor-registry-688894c58d-72txm        2/2     Running   0          7m11s
    harbor-trivy-0                          1/1     Running   0          7m11s
  13. Troubleshoot Harbor installation, if necessary. See Troubleshoot Harbor Registry Deployment.

Configure DNS for Harbor Using an Envoy Service of Type LoadBalancer (NSX-T Networking)

If the prerequisite Envoy service is exposed via a LoadBalancer, obtain the external IP address of the load balancer and create DNS records for the Harbor FQDNs.
  1. Get the External-IP address for the Envoy service of type LoadBalancer.
    kubectl get service envoy -n tanzu-system-ingress
    You should see the External-IP address returned, for example:
    NAME    TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
    envoy   LoadBalancer   10.99.25.220   10.195.141.17   80:30437/TCP,443:30589/TCP   3h27m
    Alternatively you can get the External-IP address using the following command.
    kubectl get svc envoy -n tanzu-system-ingress -o jsonpath='{.status.loadBalancer.ingress[0]}'
  2. To verify the installation of the Harbor extension, update your local /etc/hosts file with the Harbor and Notary FQDNs mapped to the External-IP address of the load balancer, for example:
    127.0.0.1 localhost
    127.0.1.1 ubuntu
    # TKGS Harbor with Envoy Load Balancer IP
    10.195.141.17 core.harbor.domain
    10.195.141.17 core.notary.harbor.domain
    
  3. To verify the installation of the Harbor extension, log in to Harbor. See Log In to the Harbor Web Interface.
  4. Create two CNAME records on a DNS server that map the Envoy service Load Balancer External-IP address to the Harbor FQDN and the Notary FQDN.
  5. Install the External DNS extension. See Deploy and Manage the TKG Extension for External DNS Service Discovery.

Configure DNS for Harbor Using an Envoy Service of Type NodePort (vDS Networking)

If the prerequisite Envoy service is exposed via a NodePort, obtain the virtual machine IP address of a worker node and create DNS records for the Harbor FQDNs.
Note: To use NodePort, you must have specified the correct port.https value in the harbor-data-values.yaml file.
  1. Switch context to the vSphere Namespace where the cluster is provisioned.
    kubectl config use-context VSPHERE-NAMESPACE
  2. List the nodes in the cluster.
    kubectl get virtualmachines
    You should see the cluster nodes, for example:
    NAME                                            POWERSTATE   AGE
    tkgs-cluster-X-control-plane-6dgln              poweredOn    6h7m
    tkgs-cluster-X-control-plane-j6hq6              poweredOn    6h10m
    tkgs-cluster-X-control-plane-xc25f              poweredOn    6h14m
    tkgs-cluster-X-workers-9twdr-59bc54dc97-kt4cm   poweredOn    6h12m
    tkgs-cluster-X-workers-9twdr-59bc54dc97-pjptr   poweredOn    6h12m
    tkgs-cluster-X-workers-9twdr-59bc54dc97-t45mn   poweredOn    6h12m
  3. Pick one of the worker nodes and describe it using the following command.
    kubectl describe virtualmachines tkgs-cluster-X-workers-9twdr-59bc54dc97-kt4cm
  4. Locate the IP address of the virtual machine, for example Vm Ip: 10.115.22.43.
  5. To verify the installation of the Harbor extension, update your local /etc/hosts file with the Harbor and Notary FQDNs mapped to the worker node IP address, for example:
    127.0.0.1 localhost
    127.0.1.1 ubuntu
    # TKGS Harbor with Envoy NodePort
    10.115.22.43 core.harbor.domain
    10.115.22.43 core.notary.harbor.domain
    
  6. To verify the installation of the Harbor extension, log in to Harbor. See Log In to the Harbor Web Interface.
  7. Create two CNAME records on a DNS server that map the worker node IP address to the Harbor FQDN and the Notary FQDN.
  8. Install the External DNS extension. See Deploy and Manage the TKG Extension for External DNS Service Discovery.

Log In to the Harbor Web Interface

Once Harbor is installed and configured, log in and start using it.
  1. Access the Harbor Registry web interface at https://core.harbor.domain, or the hostname you used.

  2. Log in to Harbor with the username admin and the generated password that you put in the harbor-data-values.yaml file.

  3. Verify that you can access the Harbor user interface.

  4. Obtain the Harbor CA certificate.

    In the Harbor interface, select Projects > library, or create a New Project.

    Click Registry Certificate and download the Harbor CA certificate (ca.crt).

  5. Add the Harbor CA certificate into the trust store of Docker client so you can push and pull container images to and from the Harbor Registry. See Configure a Docker Client with the Embedded Harbor Registry Certificate.
  6. Refer to the Harbor documentation for details on using Harbor.

Troubleshoot Harbor Registry Deployment

If the deployment or reconciliation fails, run kubectl get pods -n tanzu-system-registry to view pod status. The harbor pods should be Running. If a pod status is ImagePullBackOff or ImageCrashLoopBackOff, the container image could not be pulled. Check the registry URL in the data values and the extension YAML files and make sure they are accurate.

Check the container logs, where name-XXXX is the unique pod name when you run kubectl get pods -A:
kubectl logs pod/harbor-XXXXX -c harbor -n tanzu-system-registry

Update the Harbor Extension

Update the Contour extension that is deployed to a Tanzu Kubernetes cluster.

  1. Get Harbor data values from the secret.
    kubectl get secret harbor-data-values -n tanzu-system-registry -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > harbor-data-values.yaml
    
  2. Update Harbor data values in harbor-data-values.yaml.
  3. Update the Harbor data values secret.
    kubectl create secret generic harbor-data-values --from-file=values.yaml=harbor-data-values.yaml -n tanzu-system-registry -o yaml --dry-run | kubectl replace -f-
    
    The Harbor extension will be reconciled with the new data values.
    Note: By default, kapp-controller will sync apps every 5 minutes. The update should take effect in 5 minutes or less. If you want the update to take effect immediately, change syncPeriod in harbor-extension.yaml to a lesser value and apply the Contour extension using kubectl apply -f harbor-extension.yaml.
  4. Check the status of the extension.
    kubectl get app harbor -n tanzu-system-registry

    The Contour app status should change to Reconcile Succeeded once Contour is updated.

  5. View detailed status and troubleshoot.
    kubectl get app harbor -n tanzu-system-registry -o yaml

Delete the Harbor Extension

Delete the Harbor extension from a Tanzu Kubernetes cluster.

Note: Complete the steps in order. Do not delete the Contour namespace and role objects before the Contour extension and app are deleted. Deleting the Contour namespace and role objects deletes the service account used by kapp-controller. If this service account is deleted before the app and extension are deleted, it can lead to system errors.
  1. Change directory to where you have downloaded the Harbor extension files.
    cd /extensions/registry/harbor/
  2. Delete the Harbor app.
    kubectl delete app harod -n tanzu-system-registry
    Expected result:
    app.kappctrl.k14s.io "harbor" deleted
  3. Verify that the Harbor app is deleted.
    kubectl get app Harbor -n tanzu-system-registry
    Expected result: The app is Not Found.
    apps.kappctrl.k14s.io "harbor" not found
  4. Delete the Registry namespace.
    Only after you have confirmed that the Harbor extension and app are fully deleted is safe to delete the namespace and role objects.
    kubectl delete -f namespace-role.yaml
    Expected result: The namespace where Harbor is deployed, and the associated role-based access control objects, are deleted.
    namespace "tanzu-system-registry" deleted
    serviceaccount "harbor-extension-sa" deleted
    role.rbac.authorization.k8s.io "harbor-extension-role" deleted
    rolebinding.rbac.authorization.k8s.io "harbor-extension-rolebinding" deleted
    clusterrole.rbac.authorization.k8s.io "harbor-extension-cluster-role" deleted
    clusterrolebinding.rbac.authorization.k8s.io "harbor-extension-cluster-rolebinding" deleted

Upgrade the Harbor Extension

If you have an existing Harbor extension deployed, you can upgrade it to the latest version.
  1. Get the Harbor configmap.
    kubectl get configmap harbor -n tanzu-system-harbor -o 'go-template={{ index .data "harbor.yaml" }}' > harbor-configmap.yaml
    
  2. Delete the existing Harbor deployment. See Delete the Harbor Extension.
  3. Deploy the Harbor extension. See Deploy the Harbor Extension.