You can use Harbor as a Supervisor Service as a local registry for container images from the NVIDIA NGC catalog or for storing validated ML models.

If you want to use the Harbor registry integration with Supervisor, you can follow these setup approaches:

Note: This documentation is based on VMware Cloud Foundation 5.2.1. For information on the VMware Private AI Foundation with NVIDIA functionality in VMware Cloud Foundation 5.2, see VMware Private AI Foundation with NVIDIA Guide for VMware Cloud Foundation 5.2.
Disconnected environment
Note: The installation of the Harbor service in the Supervisor requires an Internet connection.
  • Use a Harbor registry only in the Supervisor in the GPU-enabled workload domain. Perform the following tasks:
    1. Enable Harbor as a Supervisor Service.
      1. For VMware Cloud Foundation 5.2.1, log in to the vCenter Server instance for the management domain at https://<vcenter_server_fqdn>/ui.
      2. In the vSphere Client side panel, click Private AI Foundation.
      3. In the Private AI Foundation workflow, select Cloud disconnected at the top and click the Set Up a Workload Domain section.
      4. Install the Supervisor Services for Contour and Harbor. See Enable Harbor as a Supervisor Service. The wizards for installing the Contour and Harbor Supervisor Services in the guided deployment workflow are the same as in the Workload Management area of the vSphere Client.
    2. Upload AI Container Images to a Private Harbor Registry in VMware Private AI Foundation with NVIDIA

    In a disconnected environment, you can disconnect the environment from the Internet and start using the Harbor service as a local container registry after you install the service or after you install it and download the initial set of required container images. In this approach, you must manually download container images from the NVIDIA NGC catalog to a machine in the environment and then upload them to the registry.

  • Use a Harbor registry that is as a replica of an Internet-connected Harbor registry.

    One Harbor registry, running outside the VMware Private AI Foundation with NVIDIA environment, is always connected to the Internet. The Harbor registry in the Supervisor for the GPU-enabled workload domain receives container images from the connected one using a proxy mechanism. In this way, the main components of the VMware Cloud Foundation instance remain isolated.

    In this approach, additional resources are required for the connected registry.

Note: Allocate enough storage space for hosting the NVIDIA NGC containers you plan to deploy on a deep learning VM or on a TKG cluster. Accommodate at least three versions of each container in the storage space.

If connecting to the Internet while installing the Harbor service or setting up a connected Harbor registry is not an option for your organization, use a container registry by another vendor.

Storing ML models
Use a Harbor registry in the Supervisor in the GPU-enabled workload domain. Perform the following tasks:
  1. See Enable Harbor as a Supervisor Service.
  2. Upload AI Container Images to a Private Harbor Registry in VMware Private AI Foundation with NVIDIA

Upload AI Container Images to a Private Harbor Registry in VMware Private AI Foundation with NVIDIA

In a disconnected environment, where you use a Harbor registry only on the AI-ready Supervisor, you must manually upload the AI container images that you intend to deploy on a deep learning VM or a TKG cluster from the NVIDIA NGC catalog to Harbor.

Procedure

  1. On the machines for access to NVIDIA NGC and to the disconnected VMware Cloud Foundation instance, configure the Docker client with the certificate of the Harbor registry.
    See Configure a Docker Client with a Registry Certificate.
  2. Log in to NVIDA NGC.
    Use the reserved user name of $oauthtoken and paste the API key in the password field. 
    docker login nvcr.io
  3. Pull the required container images to the machine with access to NVIDIA NGC catalog and save them to an archive.
    For example, to download the CUDA Sample container image, run the following commands. 
    docker pull nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8
docker save > cuda-sample.tar nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8
  4. Copy the archive to the machine with access to the local container registry.
  5. On the machine with access to the local container registry, load the container image. 
    docker load < cuda-sample.tar
  6. Log in to the Harbor registry.
    For example, if the Harbor registry is running at my-harbor-registry.example.com, run the following commands. 
    docker login my-harbor-registry.example.com
  7. Tag the image that you want to push to the project with the same name as the namespace where you want to use it.
    For example, to tag the CUDA Sample container image as latest for the my-private-ai-namespace project on the my-harbor-registry.example.com registry, run the following command. 
    docker tag nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8 my-harbor-registry.example.com/my-private-ai-namespace/cuda-sample:latest
  8. Push the container images to the Harbor registry. 
    docker push my-harbor-registry.example.com/my-private-ai-namespace/cuda-sample:latest

Create a Harbor Registry in VMware Private AI Foundation with NVIDIA as a Replica of a Connected Registry

To be able to update easily to the latest images in the NVIDIA NGC catalog, you can use a Harbor registry in a Supervisor that is in another VI workload domain or VMware Cloud Foundation instance and can be connected to Internet. You then replicate this connected registry on the Supervisor where you plan to run AI workloads.

You pull the latest container images from NVIDIA NGC to the connected Harbor registry and transfer them to the disconnected one by using a proxy-cached connection. In this way, you do not have to download container images and then upload them manually on a frequent basis.

Note: You can also use a connected container registry by another vendor.

You set up the network between the two registries in the following way:

  • The connected registry is routable to the replica registry.
  • The connected registry is placed in a DMZ where only docker push and docker pull communication is allowed between the two registries.

Prerequisites

Enable Harbor as a Supervisor Service in the Supervisor in the GPU-enabled workload domain.

Procedure

  1. Log in to the connected Harbor Registry UI as a Harbor system administrator.
  2. Go to the Administration > Registries page and create an endpoint for the NVIDIA NGC catalog nvcr.io/nvaie selecting the Docker Registry provider and with your NVIDIA NGC API key.
  3. Go to the Administration > Projects page and create a proxy-cache project, connected to the endpoint for nvcr.io/nvaie.
  4. Back on the Registries page, create a replication endpoint for the disconnected registry, selecting the Harbor provider.
  5. Go to the Administration > Replications page and create a replication rule.
    • Use push-based replication mode.
    • In the Destination registry property, enter the URL of the disconnected registry on the AI-ready Supervisor.
    • Set filters, target namespace and trigger mode according to the requirements of your organization.

What to do next

  1. Pull the container images that are required by your organization from NVIDIA NGC to the connected registry by running docker pull on the Docker client machine.
  2. If the replication rule has manual trigger mode, run manually replications as needed.