Provision a GPU-Accelerated TKG Cluster by Using a Self-Service Catalog in VMware Private AI Foundation with NVIDIA

In VMware Private AI Foundation with NVIDIA, as a DevOps engineer, you can provision a TKG cluster accelerated with NVIDIA GPUs from VMware Aria Automation by using an AI Kubernetes Cluster self-service catalog items in Automation Service Broker. Then, you can deploy AI container images from NVIDIA NGC on the cluster.

Prerequisites

Verify with your cloud administrator that VMware Private AI Foundation with NVIDIA is configured. See Preparing VMware Cloud Foundation for Private AI Workload Deployment.

Procedure

• ♦ In Automation Service Broker, deploy an AI Kubernetes Cluster catalog item on the Supervisor instance configured by the cloud administrator.
  • For a non-RAG Tanzu Grid Kubernetes cluster, use the AI Kubernetes Cluster catalog item. See Deploy a GPU-accelerated Tanzu Kubernetes Grid cluster.
  • For a RAG-based Tanzu Grid Kubernetes Grid cluster, use the AI Kubernetes RAG Cluster catalog item. See Deploy a GPU-accelerated Tanzu Kubernetes Grid RAG cluster.

What to do next

Run an AI container image. In a connected environment, use the NVIDIA NGC catalog. In a disconnected environment, use the Harbor Registry on the Supervisor.

For a RAG-based Tanzu Grid Kubernetes Grid cluster, deploy a pgvector PostgreSQL database in VMware Data Services Manager and install the RAG Sample Pipeline from NVIDIA. See Deploy a RAG Workload on a TKG Cluster.