Personalizzazione della distribuzione di Deep Learning VM in VMware Private AI Foundation with NVIDIA

Quando si distribuisce un'istanza di Deep Learning VM in vSphere IaaS control plane utilizzando kubectl o direttamente in un cluster vSphere, è necessario compilare le proprietà personalizzate della macchina virtuale.

Per informazioni sulle immagini di Deep Learning VM in VMware Private AI Foundation with NVIDIA, vedere Informazioni sulle immagini delle macchine virtuali di deep learning in VMware Private AI Foundation with NVIDIA.

Proprietà OVF delle macchine virtuali di deep learning

Quando si distribuisce un'istanza di Deep Learning VM, è necessario compilare le proprietà della macchina virtuale personalizzate per automatizzare la configurazione del sistema operativo Linux, la distribuzione del driver guest della vGPU, nonché la distribuzione e la configurazione dei container NGC per i carichi di lavoro DL.

L'immagine di Deep Learning VM più recente ha le proprietà OVF seguenti:

Categoria	Parametro	Etichetta in vSphere Client	Descrizione
Proprietà del sistema operativo di base	instance-id	ID istanza	Obbligatorio. ID istanza univoco per la macchina virtuale. Un ID istanza identifica in modo univoco un'istanza. Quando un ID istanza viene modificato, cloud-init gestisce l'istanza come una nuova istanza ed esegue nuovamente il processo cloud-init.
	hostname	Nome host	Obbligatorio. Nome host dell'appliance.
	seedfrom	URL da cui effettuare il seeding dei dati dell'istanza	Facoltativo. URL da cui estrarre il valore del parametro user-data e dei metadati.
	public-keys	Chiave pubblica SSH	Se specificato, l'istanza popola il valore authorized_keys di SSH dell'utente predefinito con questo valore.
	user-data	Dati utente codificati	Un set di script o altri metadati che viene inserito nella macchina virtuale al momento del provisioning. Questa proprietà rappresenta il contenuto effettivo dello script cloud-init. Questo valore deve essere codificato tramite base64. È possibile utilizzare questa proprietà per specificare il container del carico di lavoro DL che si desidera distribuire, ad esempio PyTorch o TensorFlow. Vedere Carichi di lavoro di deep learning in VMware Private AI Foundation with NVIDIA. Questa proprietà consente di impostare un indirizzo IP statico per una macchina virtuale distribuita direttamente in un cluster vSphere. Vedere Assegnazione di un indirizzo IP statico a una macchina virtuale di deep learning in VMware Private AI Foundation with NVIDIA.
	password	Password utente predefinito	Obbligatorio. Password dell'account utente vmware predefinito.
Installazione del driver vGPU	vgpu-license	Licenza vGPU	Obbligatorio. Token di configurazione del client NVIDIA vGPU. Il token viene salvato nel file /etc/nvidia/ClientConfigToken/client_configuration_token.tok.
	nvidia-portal-api-key	Chiave API portale NVIDIA	Obbligatorio in un ambiente connesso. Chiave API scaricata dal portale delle licenze NVIDIA. La chiave è necessaria per l'installazione del driver guest della vGPU.
	vgpu-fallback-version	Versione driver host vGPU	Installa direttamente questa versione del driver guest della vGPU.
	vgpu-url	URL per i download della vGPU air gap	Obbligatorio in un ambiente disconnesso. URL da cui scaricare il driver guest della vGPU. Per informazioni sulla configurazione necessaria del server Web locale, vedere Preparazione di VMware Cloud Foundation per la distribuzione del carico di lavoro di Private AI.
Automazione del carico di lavoro DL	registry-uri	URI registro	Obbligatorio in un ambiente disconnesso o se si intende utilizzare un registro di container privato per evitare di scaricare immagini da Internet. URI di un registro di container privato con le immagini del container del carico di lavoro di deep learning. Obbligatorio se si fa riferimento a un registro privato in user-data o image-oneliner.
	registry-user	Nome utente registro	Obbligatorio se si utilizza un registro di container privato che richiede l'autenticazione di base.
	registry-passwd	Password registro	Obbligatorio se si utilizza un registro di container privato che richiede l'autenticazione di base.
	registry-2-uri	URI registro secondario	Obbligatorio se si utilizza un secondo registro di container privato basato su Docker che richiede l'autenticazione di base. Ad esempio, quando si distribuisce un'istanza di Deep Learning VM con il carico di lavoro DL di NVIDIA RAG preinstallato, un'immagine di pgvector viene scaricata da Docker Hub. È possibile utilizzare i parametri registry-2- per ignorare un limite di velocità pull per docker.io.
	registry-2-user	Nome utente registro secondario	Obbligatorio se si utilizza un secondo registro di container privato.
	registry-2-passwd	Password registro secondario	Obbligatorio se si utilizza un secondo registro di container privato.
	image-oneliner	Comando a una riga codificato	Comando bash a una riga che viene eseguito al momento del provisioning della macchina virtuale. Questo valore deve essere codificato tramite base64. È possibile utilizzare questa proprietà per specificare il container del carico di lavoro DL che si desidera distribuire, ad esempio PyTorch o TensorFlow. Vedere Carichi di lavoro di deep learning in VMware Private AI Foundation with NVIDIA. Attenzione: Evitare di utilizzare sia user-data sia image-oneliner.
	docker-compose-uri	File di composizione Docker codificato	Obbligatorio se è necessario un file di composizione di Docker per avviare il container di carichi di lavoro DL. Contenuti del file docker-compose.yaml che verranno inseriti nella macchina virtuale al momento del provisioning dopo l'avvio della macchina virtuale con la GPU abilitata. Questo valore deve essere codificato tramite base64.
	config-json	config.json codificato	Contenuti di un file di configurazione per l'aggiunta di dettagli per i server proxy. Questo valore deve essere codificato tramite base64. Vedere Configurazione di un'stanza di Deep Learning VM con un server proxy.
	conda-environment-install	Installazione dell'ambiente Conda	Elenco di ambienti Conda separati da virgole da installare automaticamente al termine della distribuzione della macchina virtuale. Ambienti disponibili: pytorch2.3_py3.12

Carichi di lavoro di deep learning in VMware Private AI Foundation with NVIDIA

È possibile eseguire il provisioning di una macchina virtuale di deep learning con un carico di lavoro di deep learning (DL) supportato oltre ai suoi componenti incorporati. I carichi di lavoro DL vengono scaricati dal catalogo NVIDIA NGC e sono ottimizzati per la GPU e convalidati da NVIDIA e VMware by Broadcom.

Per una panoramica delle immagini delle macchine virtuali di deep learning, vedere Informazioni sulle immagini delle macchine virtuali di deep learning in VMware Private AI Foundation with NVIDIA.

Esempio di CUDA

È possibile utilizzare una macchina virtuale di deep learning con esempi CUDA in esecuzione per esplorare l'aggiunta di un vettore, la simulazione gravitazionale di n-corpi o altri esempi in una macchina virtuale. Vedere la pagina Esempi di CUDA.

Dopo l'avvio, la macchina virtuale di deep learning esegue un carico di lavoro di esempio CUDA per testare il driver guest della vGPU. È possibile esaminare l'output del test nel file /var/log/dl.log.

Tabella 1. Immagine del container di esempio CUDA

Componente	Descrizione
Immagine del container	nvcr.io/nvidia/k8s/cuda-sample:ngc_image_tag Ad esempio: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8 Per informazioni sulle immagini di container di esempio CUDA supportate per le macchine virtuali di deep learning, vedere Note di rilascio di VMware Deep Learning VM.
Input necessari	Per distribuire un carico di lavoro di esempio CUDA, è necessario impostare le proprietà OVF per la macchina virtuale di deep learning nel modo seguente: Utilizzare una delle proprietà seguenti specifiche per l'immagine dell'esempio CUDA. Script cloud-init. Codificarlo nel formato base64. #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh set_proxy "http" "https" "socks5" trap 'error_exit "Unexpected error occurs at dl workload"' ERR DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d $REGISTRY_URI_PATH/nvidia/k8s/cuda-sample:ngc_image_tag - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Ad esempio, per vectoradd-cuda11.7.1-ubi8, specificare lo script seguente in formato base64: I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBzZXQgLWV1CiAgICBzb3VyY2UgL29wdC9kbHZtL3V0aWxzLnNoCiAgICBzZXRfcHJveHkgImh0dHAiICJodHRwcyIgInNvY2tzNSIKICAgIHRyYXAgJ2Vycm9yX2V4aXQgIlVuZXhwZWN0ZWQgZXJyb3Igb2NjdXJzIGF0IGRsIHdvcmtsb2FkIicgRVJSCiAgICBERUZBVUxUX1JFR19VUkk9Im52Y3IuaW8iCiAgICBSRUdJU1RSWV9VUklfUEFUSD0kKGdyZXAgcmVnaXN0cnktdXJpIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKCiAgICBpZiBbWyAteiAiJFJFR0lTVFJZX1VSSV9QQVRIIiBdXTsgdGhlbgogICAgICAjIElmIFJFR0lTVFJZX1VSSV9QQVRIIGlzIG51bGwgb3IgZW1wdHksIHVzZSB0aGUgZGVmYXVsdCB2YWx1ZQogICAgICBSRUdJU1RSWV9VUklfUEFUSD0kREVGQVVMVF9SRUdfVVJJCiAgICAgIGVjaG8gIlJFR0lTVFJZX1VSSV9QQVRIIHdhcyBlbXB0eS4gVXNpbmcgZGVmYXVsdDogJFJFR0lTVFJZX1VSSV9QQVRIIgogICAgZmkKICAgIAogICAgIyBJZiBSRUdJU1RSWV9VUklfUEFUSCBjb250YWlucyAnLycsIGV4dHJhY3QgdGhlIFVSSSBwYXJ0CiAgICBpZiBbWyAkUkVHSVNUUllfVVJJX1BBVEggPT0gKiIvIiogXV07IHRoZW4KICAgICAgUkVHSVNUUllfVVJJPSQoZWNobyAiJFJFR0lTVFJZX1VSSV9QQVRIIiB8IGN1dCAtZCcvJyAtZjEpCiAgICBlbHNlCiAgICAgIFJFR0lTVFJZX1VSST0kUkVHSVNUUllfVVJJX1BBVEgKICAgIGZpCiAgCiAgICBSRUdJU1RSWV9VU0VSTkFNRT0kKGdyZXAgcmVnaXN0cnktdXNlciAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpCiAgICBSRUdJU1RSWV9QQVNTV09SRD0kKGdyZXAgcmVnaXN0cnktcGFzc3dkIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKICAgIGlmIFtbIC1uICIkUkVHSVNUUllfVVNFUk5BTUUiICYmIC1uICIkUkVHSVNUUllfUEFTU1dPUkQiIF1dOyB0aGVuCiAgICAgIGRvY2tlciBsb2dpbiAtdSAkUkVHSVNUUllfVVNFUk5BTUUgLXAgJFJFR0lTVFJZX1BBU1NXT1JEICRSRUdJU1RSWV9VUkkKICAgIGVsc2UKICAgICAgZWNobyAiV2FybmluZzogdGhlIHJlZ2lzdHJ5J3MgdXNlcm5hbWUgYW5kIHBhc3N3b3JkIGFyZSBpbnZhbGlkLCBTa2lwcGluZyBEb2NrZXIgbG9naW4uIgogICAgZmkKICAgIAogICAgZG9ja2VyIHJ1biAtZCAkUkVHSVNUUllfVVJJX1BBVEgvbnZpZGlhL2s4cy9jdWRhLXNhbXBsZTp2ZWN0b3JhZGQtY3VkYTExLjcuMS11Ymk4CgotIHBhdGg6IC9vcHQvZGx2bS91dGlscy5zaAogIHBlcm1pc3Npb25zOiAnMDc1NScKICBjb250ZW50OiB8CiAgICAjIS9iaW4vYmFzaAogICAgZXJyb3JfZXhpdCgpIHsKICAgICAgZWNobyAiRXJyb3I6ICQxIiA+JjIKICAgICAgdm10b29sc2QgLS1jbWQgImluZm8tc2V0IGd1ZXN0aW5mby52bXNlcnZpY2UuYm9vdHN0cmFwLmNvbmRpdGlvbiBmYWxzZSwgRExXb3JrbG9hZEZhaWx1cmUsICQxIgogICAgICBleGl0IDEKICAgIH0KCiAgICBjaGVja19wcm90b2NvbCgpIHsKICAgICAgbG9jYWwgcHJveHlfdXJsPSQxCiAgICAgIHNoaWZ0CiAgICAgIGxvY2FsIHN1cHBvcnRlZF9wcm90b2NvbHM9KCIkQCIpCiAgICAgIGlmIFtbIC1uICIke3Byb3h5X3VybH0iIF1dOyB0aGVuCiAgICAgICAgbG9jYWwgcHJvdG9jb2w9JChlY2hvICIke3Byb3h5X3VybH0iIHwgYXdrIC1GICc6Ly8nICd7aWYgKE5GID4gMSkgcHJpbnQgJDE7IGVsc2UgcHJpbnQgIiJ9JykKICAgICAgICBpZiBbIC16ICIkcHJvdG9jb2wiIF07IHRoZW4KICAgICAgICAgIGVjaG8gIk5vIHNwZWNpZmljIHByb3RvY29sIHByb3ZpZGVkLiBTa2lwcGluZyBwcm90b2NvbCBjaGVjay4iCiAgICAgICAgICByZXR1cm4gMAogICAgICAgIGZpCiAgICAgICAgbG9jYWwgcHJvdG9jb2xfaW5jbHVkZWQ9ZmFsc2UKICAgICAgICBmb3IgdmFyIGluICIke3N1cHBvcnRlZF9wcm90b2NvbHNbQF19IjsgZG8KICAgICAgICAgIGlmIFtbICIke3Byb3RvY29sfSIgPT0gIiR7dmFyfSIgXV07IHRoZW4KICAgICAgICAgICAgcHJvdG9jb2xfaW5jbHVkZWQ9dHJ1ZQogICAgICAgICAgICBicmVhawogICAgICAgICAgZmkKICAgICAgICBkb25lCiAgICAgICAgaWYgW1sgIiR7cHJvdG9jb2xfaW5jbHVkZWR9IiA9PSBmYWxzZSBdXTsgdGhlbgogICAgICAgICAgZXJyb3JfZXhpdCAiVW5zdXBwb3J0ZWQgcHJvdG9jb2w6ICR7cHJvdG9jb2x9LiBTdXBwb3J0ZWQgcHJvdG9jb2xzIGFyZTogJHtzdXBwb3J0ZWRfcHJvdG9jb2xzWypdfSIKICAgICAgICBmaQogICAgICBmaQogICAgfQoKICAgICMgJEA6IGxpc3Qgb2Ygc3VwcG9ydGVkIHByb3RvY29scwogICAgc2V0X3Byb3h5KCkgewogICAgICBsb2NhbCBzdXBwb3J0ZWRfcHJvdG9jb2xzPSgiJEAiKQoKICAgICAgQ09ORklHX0pTT05fQkFTRTY0PSQoZ3JlcCAnY29uZmlnLWpzb24nIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKICAgICAgQ09ORklHX0pTT049JChlY2hvICR7Q09ORklHX0pTT05fQkFTRTY0fSB8IGJhc2U2NCAtLWRlY29kZSkKCiAgICAgIEhUVFBfUFJPWFlfVVJMPSQoZWNobyAiJHtDT05GSUdfSlNPTn0iIHwganEgLXIgJy5odHRwX3Byb3h5IC8vIGVtcHR5JykKICAgICAgSFRUUFNfUFJPWFlfVVJMPSQoZWNobyAiJHtDT05GSUdfSlNPTn0iIHwganEgLXIgJy5odHRwc19wcm94eSAvLyBlbXB0eScpCiAgICAgIGlmIFtbICQ/IC1uZSAwIHx8ICgteiAiJHtIVFRQX1BST1hZX1VSTH0iICYmIC16ICIke0hUVFBTX1BST1hZX1VSTH0iKSBdXTsgdGhlbgogICAgICAgIGVjaG8gIkluZm86IFRoZSBjb25maWctanNvbiB3YXMgcGFyc2VkLCBidXQgbm8gcHJveHkgc2V0dGluZ3Mgd2VyZSBmb3VuZC4iCiAgICAgICAgcmV0dXJuIDAKICAgICAgZmkKICAKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUF9QUk9YWV9VUkx9IiAiJHtzdXBwb3J0ZWRfcHJvdG9jb2xzW0BdfSIKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUFNfUFJPWFlfVVJMfSIgIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iCgogICAgICBpZiAhIGdyZXAgLXEgJ2h0dHBfcHJveHknIC9ldGMvZW52aXJvbm1lbnQ7IHRoZW4KICAgICAgICBlY2hvICJleHBvcnQgaHR0cF9wcm94eT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBodHRwc19wcm94eT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgSFRUUF9QUk9YWT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBIVFRQU19QUk9YWT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgbm9fcHJveHk9bG9jYWxob3N0LDEyNy4wLjAuMSIgPj4gL2V0Yy9lbnZpcm9ubWVudAogICAgICAgIHNvdXJjZSAvZXRjL2Vudmlyb25tZW50CiAgICAgIGZpCiAgICAgIAogICAgICAjIENvbmZpZ3VyZSBEb2NrZXIgdG8gdXNlIGEgcHJveHkKICAgICAgbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCiAgICAgIGVjaG8gIltTZXJ2aWNlXQogICAgICBFbnZpcm9ubWVudD1cIkhUVFBfUFJPWFk9JHtIVFRQX1BST1hZX1VSTH1cIgogICAgICBFbnZpcm9ubWVudD1cIkhUVFBTX1BST1hZPSR7SFRUUFNfUFJPWFlfVVJMfVwiCiAgICAgIEVudmlyb25tZW50PVwiTk9fUFJPWFk9bG9jYWxob3N0LDEyNy4wLjAuMVwiIiA+IC9ldGMvc3lzdGVtZC9zeXN0ZW0vZG9ja2VyLnNlcnZpY2UuZC9wcm94eS5jb25mCiAgICAgIHN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCiAgICAgIHN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgoKICAgICAgZWNobyAiSW5mbzogZG9ja2VyIGFuZCBzeXN0ZW0gZW52aXJvbm1lbnQgYXJlIG5vdyBjb25maWd1cmVkIHRvIHVzZSB0aGUgcHJveHkgc2V0dGluZ3MiCiAgICB9 che corrisponde allo script seguente in formato testo normale: #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh set_proxy "http" "https" "socks5" trap 'error_exit "Unexpected error occurs at dl workload"' ERR DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d $REGISTRY_URI_PATH/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8 - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Immagine one-liner. Codificarlo nel formato base64 docker run -d nvcr.io/nvidia/k8s/cuda-sample:ngc_image_tag Ad esempio, per vectoradd-cuda11.7.1-ubi8, specificare lo script seguente in formato base64: ZG9ja2VyIHJ1biAtZCBudmNyLmlvL252aWRpYS9rOHMvY3VkYS1zYW1wbGU6dmVjdG9yYWRkLWN1ZGExMS43LjEtdWJpOA== che corrisponde allo script seguente in formato testo normale: docker run -d nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8 Immettere le proprietà di installazione del driver guest vGPU, ad esempio vgpu-license e nvidia-portal-api-key. Specificare i valori per le proprietà necessarie per un ambiente disconnesso in base alle esigenze. Vedere Proprietà OVF delle macchine virtuali di deep learning.
Output	Registri di installazione per il driver guest della vGPU in /var/log/vgpu-install.log. Per verificare che il driver guest della vGPU sia installato e che la licenza sia allocata, eseguire il comando seguente: nvidia-smi -q |grep -i license Registri dello script cloud-init in /var/log/dl.log.

PyTorch

È possibile utilizzare una macchina virtuale di deep learning con una libreria PyTorch per esplorare AI conversazionale, l'elaborazione del linguaggio naturale (NLP) e altri tipi di modelli AI in una macchina virtuale. Vedere la pagina di PyTorch.

Dopo l'avvio, la macchina virtuale di deep learning avvia un'istanza di JupyterLab con i pacchetti PyTorch installati e configurati.

Tabella 2. Immagine del container PyTorch

Componente	Descrizione
Immagine del container	nvcr.io/nvidia/pytorch:ngc_image_tag Ad esempio: nvcr.io/nvidia/pytorch:23.10-py3 Per informazioni sulle immagini dei container PyTorch supportate per le macchine virtuali di deep learning, vedere Note di rilascio di VMware Deep Learning VM.
Input necessari	Per distribuire un carico di lavoro PyTorch, è necessario impostare le proprietà OVF per la macchina virtuale di deep learning nel modo seguente: Utilizzare una delle proprietà seguenti specifiche per l'immagine PyTorch. Script cloud-init. Codificarlo nel formato base64. #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all -p 8888:8888 $REGISTRY_URI_PATH/nvidia/pytorch:ngc_image_tag /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Ad esempio, per pytorch:23.10-py3, specificare lo script seguente in formato base 64: I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBzZXQgLWV1CiAgICBzb3VyY2UgL29wdC9kbHZtL3V0aWxzLnNoCiAgICB0cmFwICdlcnJvcl9leGl0ICJVbmV4cGVjdGVkIGVycm9yIG9jY3VycyBhdCBkbCB3b3JrbG9hZCInIEVSUgogICAgc2V0X3Byb3h5ICJodHRwIiAiaHR0cHMiICJzb2NrczUiCgogICAgREVGQVVMVF9SRUdfVVJJPSJudmNyLmlvIgogICAgUkVHSVNUUllfVVJJX1BBVEg9JChncmVwIHJlZ2lzdHJ5LXVyaSAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpCgogICAgaWYgW1sgLXogIiRSRUdJU1RSWV9VUklfUEFUSCIgXV07IHRoZW4KICAgICAgIyBJZiBSRUdJU1RSWV9VUklfUEFUSCBpcyBudWxsIG9yIGVtcHR5LCB1c2UgdGhlIGRlZmF1bHQgdmFsdWUKICAgICAgUkVHSVNUUllfVVJJX1BBVEg9JERFRkFVTFRfUkVHX1VSSQogICAgICBlY2hvICJSRUdJU1RSWV9VUklfUEFUSCB3YXMgZW1wdHkuIFVzaW5nIGRlZmF1bHQ6ICRSRUdJU1RSWV9VUklfUEFUSCIKICAgIGZpCiAgICAKICAgICMgSWYgUkVHSVNUUllfVVJJX1BBVEggY29udGFpbnMgJy8nLCBleHRyYWN0IHRoZSBVUkkgcGFydAogICAgaWYgW1sgJFJFR0lTVFJZX1VSSV9QQVRIID09ICoiLyIqIF1dOyB0aGVuCiAgICAgIFJFR0lTVFJZX1VSST0kKGVjaG8gIiRSRUdJU1RSWV9VUklfUEFUSCIgfCBjdXQgLWQnLycgLWYxKQogICAgZWxzZQogICAgICBSRUdJU1RSWV9VUkk9JFJFR0lTVFJZX1VSSV9QQVRICiAgICBmaQogIAogICAgUkVHSVNUUllfVVNFUk5BTUU9JChncmVwIHJlZ2lzdHJ5LXVzZXIgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgUkVHSVNUUllfUEFTU1dPUkQ9JChncmVwIHJlZ2lzdHJ5LXBhc3N3ZCAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpCiAgICBpZiBbWyAtbiAiJFJFR0lTVFJZX1VTRVJOQU1FIiAmJiAtbiAiJFJFR0lTVFJZX1BBU1NXT1JEIiBdXTsgdGhlbgogICAgICBkb2NrZXIgbG9naW4gLXUgJFJFR0lTVFJZX1VTRVJOQU1FIC1wICRSRUdJU1RSWV9QQVNTV09SRCAkUkVHSVNUUllfVVJJCiAgICBlbHNlCiAgICAgIGVjaG8gIldhcm5pbmc6IHRoZSByZWdpc3RyeSdzIHVzZXJuYW1lIGFuZCBwYXNzd29yZCBhcmUgaW52YWxpZCwgU2tpcHBpbmcgRG9ja2VyIGxvZ2luLiIKICAgIGZpCgogICAgZG9ja2VyIHJ1biAtZCAtLWdwdXMgYWxsIC1wIDg4ODg6ODg4OCAkUkVHSVNUUllfVVJJX1BBVEgvbnZpZGlhL3B5dG9yY2g6MjMuMTAtcHkzIC91c3IvbG9jYWwvYmluL2p1cHl0ZXIgbGFiIC0tYWxsb3ctcm9vdCAtLWlwPSogLS1wb3J0PTg4ODggLS1uby1icm93c2VyIC0tTm90ZWJvb2tBcHAudG9rZW49JycgLS1Ob3RlYm9va0FwcC5hbGxvd19vcmlnaW49JyonIC0tbm90ZWJvb2stZGlyPS93b3Jrc3BhY2UKCi0gcGF0aDogL29wdC9kbHZtL3V0aWxzLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBlcnJvcl9leGl0KCkgewogICAgICBlY2hvICJFcnJvcjogJDEiID4mMgogICAgICB2bXRvb2xzZCAtLWNtZCAiaW5mby1zZXQgZ3Vlc3RpbmZvLnZtc2VydmljZS5ib290c3RyYXAuY29uZGl0aW9uIGZhbHNlLCBETFdvcmtsb2FkRmFpbHVyZSwgJDEiCiAgICAgIGV4aXQgMQogICAgfQoKICAgIGNoZWNrX3Byb3RvY29sKCkgewogICAgICBsb2NhbCBwcm94eV91cmw9JDEKICAgICAgc2hpZnQKICAgICAgbG9jYWwgc3VwcG9ydGVkX3Byb3RvY29scz0oIiRAIikKICAgICAgaWYgW1sgLW4gIiR7cHJveHlfdXJsfSIgXV07IHRoZW4KICAgICAgICBsb2NhbCBwcm90b2NvbD0kKGVjaG8gIiR7cHJveHlfdXJsfSIgfCBhd2sgLUYgJzovLycgJ3tpZiAoTkYgPiAxKSBwcmludCAkMTsgZWxzZSBwcmludCAiIn0nKQogICAgICAgIGlmIFsgLXogIiRwcm90b2NvbCIgXTsgdGhlbgogICAgICAgICAgZWNobyAiTm8gc3BlY2lmaWMgcHJvdG9jb2wgcHJvdmlkZWQuIFNraXBwaW5nIHByb3RvY29sIGNoZWNrLiIKICAgICAgICAgIHJldHVybiAwCiAgICAgICAgZmkKICAgICAgICBsb2NhbCBwcm90b2NvbF9pbmNsdWRlZD1mYWxzZQogICAgICAgIGZvciB2YXIgaW4gIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iOyBkbwogICAgICAgICAgaWYgW1sgIiR7cHJvdG9jb2x9IiA9PSAiJHt2YXJ9IiBdXTsgdGhlbgogICAgICAgICAgICBwcm90b2NvbF9pbmNsdWRlZD10cnVlCiAgICAgICAgICAgIGJyZWFrCiAgICAgICAgICBmaQogICAgICAgIGRvbmUKICAgICAgICBpZiBbWyAiJHtwcm90b2NvbF9pbmNsdWRlZH0iID09IGZhbHNlIF1dOyB0aGVuCiAgICAgICAgICBlcnJvcl9leGl0ICJVbnN1cHBvcnRlZCBwcm90b2NvbDogJHtwcm90b2NvbH0uIFN1cHBvcnRlZCBwcm90b2NvbHMgYXJlOiAke3N1cHBvcnRlZF9wcm90b2NvbHNbKl19IgogICAgICAgIGZpCiAgICAgIGZpCiAgICB9CgogICAgIyAkQDogbGlzdCBvZiBzdXBwb3J0ZWQgcHJvdG9jb2xzCiAgICBzZXRfcHJveHkoKSB7CiAgICAgIGxvY2FsIHN1cHBvcnRlZF9wcm90b2NvbHM9KCIkQCIpCgogICAgICBDT05GSUdfSlNPTl9CQVNFNjQ9JChncmVwICdjb25maWctanNvbicgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgICBDT05GSUdfSlNPTj0kKGVjaG8gJHtDT05GSUdfSlNPTl9CQVNFNjR9IHwgYmFzZTY0IC0tZGVjb2RlKQoKICAgICAgSFRUUF9QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBfcHJveHkgLy8gZW1wdHknKQogICAgICBIVFRQU19QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBzX3Byb3h5IC8vIGVtcHR5JykKICAgICAgaWYgW1sgJD8gLW5lIDAgfHwgKC16ICIke0hUVFBfUFJPWFlfVVJMfSIgJiYgLXogIiR7SFRUUFNfUFJPWFlfVVJMfSIpIF1dOyB0aGVuCiAgICAgICAgZWNobyAiSW5mbzogVGhlIGNvbmZpZy1qc29uIHdhcyBwYXJzZWQsIGJ1dCBubyBwcm94eSBzZXR0aW5ncyB3ZXJlIGZvdW5kLiIKICAgICAgICByZXR1cm4gMAogICAgICBmaQoKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUF9QUk9YWV9VUkx9IiAiJHtzdXBwb3J0ZWRfcHJvdG9jb2xzW0BdfSIKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUFNfUFJPWFlfVVJMfSIgIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iCgogICAgICBpZiAhIGdyZXAgLXEgJ2h0dHBfcHJveHknIC9ldGMvZW52aXJvbm1lbnQ7IHRoZW4KICAgICAgICBlY2hvICJleHBvcnQgaHR0cF9wcm94eT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBodHRwc19wcm94eT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgSFRUUF9QUk9YWT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBIVFRQU19QUk9YWT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgbm9fcHJveHk9bG9jYWxob3N0LDEyNy4wLjAuMSIgPj4gL2V0Yy9lbnZpcm9ubWVudAogICAgICAgIHNvdXJjZSAvZXRjL2Vudmlyb25tZW50CiAgICAgIGZpCiAgICAgIAogICAgICAjIENvbmZpZ3VyZSBEb2NrZXIgdG8gdXNlIGEgcHJveHkKICAgICAgbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCiAgICAgIGVjaG8gIltTZXJ2aWNlXQogICAgICBFbnZpcm9ubWVudD1cIkhUVFBfUFJPWFk9JHtIVFRQX1BST1hZX1VSTH1cIgogICAgICBFbnZpcm9ubWVudD1cIkhUVFBTX1BST1hZPSR7SFRUUFNfUFJPWFlfVVJMfVwiCiAgICAgIEVudmlyb25tZW50PVwiTk9fUFJPWFk9bG9jYWxob3N0LDEyNy4wLjAuMVwiIiA+IC9ldGMvc3lzdGVtZC9zeXN0ZW0vZG9ja2VyLnNlcnZpY2UuZC9wcm94eS5jb25mCiAgICAgIHN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCiAgICAgIHN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgoKICAgICAgZWNobyAiSW5mbzogZG9ja2VyIGFuZCBzeXN0ZW0gZW52aXJvbm1lbnQgYXJlIG5vdyBjb25maWd1cmVkIHRvIHVzZSB0aGUgcHJveHkgc2V0dGluZ3MiCiAgICB9 che corrisponde allo script seguente in formato testo normale. #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all -p 8888:8888 $REGISTRY_URI_PATH/nvidia/pytorch:23.10-py3 /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Immagine one-liner. Codificarlo nel formato base64. docker run -d -p 8888:8888 nvcr.io/nvidia/pytorch:ngc_image_tag /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace Ad esempio, per pytorch:23.10-py3, specificare lo script seguente in formato base 64: ZG9ja2VyIHJ1biAtZCAtcCA4ODg4Ojg4ODggbnZjci5pby9udmlkaWEvcHl0b3JjaDoyMy4xMC1weTMgL3Vzci9sb2NhbC9iaW4vanVweXRlciBsYWIgLS1hbGxvdy1yb290IC0taXA9KiAtLXBvcnQ9ODg4OCAtLW5vLWJyb3dzZXIgLS1Ob3RlYm9va0FwcC50b2tlbj0nJyAtLU5vdGVib29rQXBwLmFsbG93X29yaWdpbj0nKicgLS1ub3RlYm9vay1kaXI9L3dvcmtzcGFjZQ== che corrisponde allo script seguente in formato testo normale: docker run -d -p 8888:8888 nvcr.io/nvidia/pytorch:23.10-py3 /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace Immettere le proprietà di installazione del driver guest vGPU, ad esempio vgpu-license e nvidia-portal-api-key. Specificare i valori per le proprietà necessarie per un ambiente disconnesso in base alle esigenze. Vedere Proprietà OVF delle macchine virtuali di deep learning.
Output	Registri di installazione per il driver guest della vGPU in /var/log/vgpu-install.log. Per verificare che il driver guest della vGPU sia installato, eseguire il comando nvidia-smi. Registri dello script cloud-init in /var/log/dl.log. Container PyTorch. Per verificare che il container PyTorch sia in esecuzione, eseguire i comandi sudo docker ps -a e sudo docker logs container_id. Istanza di JupyterLab accessibile all'indirizzo http://dl_vm_ip:8888 Nel terminale di JupyterLab, verificare che nel notebook siano disponibili le funzionalità seguenti: Per verificare che JupyterLab possa accedere alla risorsa vGPU, eseguire nvidia-smi. Per verificare che i pacchetti relativi a PyTorch siano installati, eseguire pip show.

TensorFlow

È possibile utilizzare una macchina virtuale di deep learning con una libreria TensorFlow per esplorare AI conversazionale, l'elaborazione del linguaggio naturale (NLP) e altri tipi di modelli AI in una macchina virtuale. Vedere la pagina di TensorFlow.

Dopo l'avvio, la macchina virtuale di deep learning avvia un'istanza di JupyterLab con i pacchetti TensorFlow installati e configurati.

Tabella 3. Immagine del container TensorFlow

Componente	Descrizione
Immagine del container	nvcr.io/nvidia/tensorflow:ngc_image_tag Ad esempio: nvcr.io/nvidia/tensorflow:23.10-tf2-py3 Per informazioni sulle immagini del container TensorFlow supportate per le macchine virtuali di deep learning, vedere Note di rilascio di VMware Deep Learning VM.
Input necessari	Per distribuire un carico di lavoro TensorFlow, è necessario impostare le proprietà OVF per la macchina virtuale di deep learning nel modo seguente: Utilizzare una delle proprietà seguenti specifiche per l'immagine di TensorFlow. Script cloud-init. Codificarlo nel formato base64. #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all -p 8888:8888 $REGISTRY_URI_PATH/nvidia/tensorflow:ngc_image_tag /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Ad esempio, per tensorflow:23.10-tf2-py3, specificare lo script seguente in formato base64: I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBzZXQgLWV1CiAgICBzb3VyY2UgL29wdC9kbHZtL3V0aWxzLnNoCiAgICB0cmFwICdlcnJvcl9leGl0ICJVbmV4cGVjdGVkIGVycm9yIG9jY3VycyBhdCBkbCB3b3JrbG9hZCInIEVSUgogICAgc2V0X3Byb3h5ICJodHRwIiAiaHR0cHMiICJzb2NrczUiCiAgICAKICAgIERFRkFVTFRfUkVHX1VSST0ibnZjci5pbyIKICAgIFJFR0lTVFJZX1VSSV9QQVRIPSQoZ3JlcCByZWdpc3RyeS11cmkgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQoKICAgIGlmIFtbIC16ICIkUkVHSVNUUllfVVJJX1BBVEgiIF1dOyB0aGVuCiAgICAgICMgSWYgUkVHSVNUUllfVVJJX1BBVEggaXMgbnVsbCBvciBlbXB0eSwgdXNlIHRoZSBkZWZhdWx0IHZhbHVlCiAgICAgIFJFR0lTVFJZX1VSSV9QQVRIPSRERUZBVUxUX1JFR19VUkkKICAgICAgZWNobyAiUkVHSVNUUllfVVJJX1BBVEggd2FzIGVtcHR5LiBVc2luZyBkZWZhdWx0OiAkUkVHSVNUUllfVVJJX1BBVEgiCiAgICBmaQogICAgCiAgICAjIElmIFJFR0lTVFJZX1VSSV9QQVRIIGNvbnRhaW5zICcvJywgZXh0cmFjdCB0aGUgVVJJIHBhcnQKICAgIGlmIFtbICRSRUdJU1RSWV9VUklfUEFUSCA9PSAqIi8iKiBdXTsgdGhlbgogICAgICBSRUdJU1RSWV9VUkk9JChlY2hvICIkUkVHSVNUUllfVVJJX1BBVEgiIHwgY3V0IC1kJy8nIC1mMSkKICAgIGVsc2UKICAgICAgUkVHSVNUUllfVVJJPSRSRUdJU1RSWV9VUklfUEFUSAogICAgZmkKICAKICAgIFJFR0lTVFJZX1VTRVJOQU1FPSQoZ3JlcCByZWdpc3RyeS11c2VyIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKICAgIFJFR0lTVFJZX1BBU1NXT1JEPSQoZ3JlcCByZWdpc3RyeS1wYXNzd2QgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgaWYgW1sgLW4gIiRSRUdJU1RSWV9VU0VSTkFNRSIgJiYgLW4gIiRSRUdJU1RSWV9QQVNTV09SRCIgXV07IHRoZW4KICAgICAgZG9ja2VyIGxvZ2luIC11ICRSRUdJU1RSWV9VU0VSTkFNRSAtcCAkUkVHSVNUUllfUEFTU1dPUkQgJFJFR0lTVFJZX1VSSQogICAgZWxzZQogICAgICBlY2hvICJXYXJuaW5nOiB0aGUgcmVnaXN0cnkncyB1c2VybmFtZSBhbmQgcGFzc3dvcmQgYXJlIGludmFsaWQsIFNraXBwaW5nIERvY2tlciBsb2dpbi4iCiAgICBmaQogICAgCiAgICBkb2NrZXIgcnVuIC1kIC0tZ3B1cyBhbGwgLXAgODg4ODo4ODg4ICRSRUdJU1RSWV9VUklfUEFUSC9udmlkaWEvdGVuc29yZmxvdzoyMy4xMC10ZjItcHkzIC91c3IvbG9jYWwvYmluL2p1cHl0ZXIgbGFiIC0tYWxsb3ctcm9vdCAtLWlwPSogLS1wb3J0PTg4ODggLS1uby1icm93c2VyIC0tTm90ZWJvb2tBcHAudG9rZW49JycgLS1Ob3RlYm9va0FwcC5hbGxvd19vcmlnaW49JyonIC0tbm90ZWJvb2stZGlyPS93b3Jrc3BhY2UKCi0gcGF0aDogL29wdC9kbHZtL3V0aWxzLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBlcnJvcl9leGl0KCkgewogICAgICBlY2hvICJFcnJvcjogJDEiID4mMgogICAgICB2bXRvb2xzZCAtLWNtZCAiaW5mby1zZXQgZ3Vlc3RpbmZvLnZtc2VydmljZS5ib290c3RyYXAuY29uZGl0aW9uIGZhbHNlLCBETFdvcmtsb2FkRmFpbHVyZSwgJDEiCiAgICAgIGV4aXQgMQogICAgfQoKICAgIGNoZWNrX3Byb3RvY29sKCkgewogICAgICBsb2NhbCBwcm94eV91cmw9JDEKICAgICAgc2hpZnQKICAgICAgbG9jYWwgc3VwcG9ydGVkX3Byb3RvY29scz0oIiRAIikKICAgICAgaWYgW1sgLW4gIiR7cHJveHlfdXJsfSIgXV07IHRoZW4KICAgICAgICBsb2NhbCBwcm90b2NvbD0kKGVjaG8gIiR7cHJveHlfdXJsfSIgfCBhd2sgLUYgJzovLycgJ3tpZiAoTkYgPiAxKSBwcmludCAkMTsgZWxzZSBwcmludCAiIn0nKQogICAgICAgIGlmIFsgLXogIiRwcm90b2NvbCIgXTsgdGhlbgogICAgICAgICAgZWNobyAiTm8gc3BlY2lmaWMgcHJvdG9jb2wgcHJvdmlkZWQuIFNraXBwaW5nIHByb3RvY29sIGNoZWNrLiIKICAgICAgICAgIHJldHVybiAwCiAgICAgICAgZmkKICAgICAgICBsb2NhbCBwcm90b2NvbF9pbmNsdWRlZD1mYWxzZQogICAgICAgIGZvciB2YXIgaW4gIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iOyBkbwogICAgICAgICAgaWYgW1sgIiR7cHJvdG9jb2x9IiA9PSAiJHt2YXJ9IiBdXTsgdGhlbgogICAgICAgICAgICBwcm90b2NvbF9pbmNsdWRlZD10cnVlCiAgICAgICAgICAgIGJyZWFrCiAgICAgICAgICBmaQogICAgICAgIGRvbmUKICAgICAgICBpZiBbWyAiJHtwcm90b2NvbF9pbmNsdWRlZH0iID09IGZhbHNlIF1dOyB0aGVuCiAgICAgICAgICBlcnJvcl9leGl0ICJVbnN1cHBvcnRlZCBwcm90b2NvbDogJHtwcm90b2NvbH0uIFN1cHBvcnRlZCBwcm90b2NvbHMgYXJlOiAke3N1cHBvcnRlZF9wcm90b2NvbHNbKl19IgogICAgICAgIGZpCiAgICAgIGZpCiAgICB9CgogICAgIyAkQDogbGlzdCBvZiBzdXBwb3J0ZWQgcHJvdG9jb2xzCiAgICBzZXRfcHJveHkoKSB7CiAgICAgIGxvY2FsIHN1cHBvcnRlZF9wcm90b2NvbHM9KCIkQCIpCgogICAgICBDT05GSUdfSlNPTl9CQVNFNjQ9JChncmVwICdjb25maWctanNvbicgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgICBDT05GSUdfSlNPTj0kKGVjaG8gJHtDT05GSUdfSlNPTl9CQVNFNjR9IHwgYmFzZTY0IC0tZGVjb2RlKQoKICAgICAgSFRUUF9QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBfcHJveHkgLy8gZW1wdHknKQogICAgICBIVFRQU19QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBzX3Byb3h5IC8vIGVtcHR5JykKICAgICAgaWYgW1sgJD8gLW5lIDAgfHwgKC16ICIke0hUVFBfUFJPWFlfVVJMfSIgJiYgLXogIiR7SFRUUFNfUFJPWFlfVVJMfSIpIF1dOyB0aGVuCiAgICAgICAgZWNobyAiSW5mbzogVGhlIGNvbmZpZy1qc29uIHdhcyBwYXJzZWQsIGJ1dCBubyBwcm94eSBzZXR0aW5ncyB3ZXJlIGZvdW5kLiIKICAgICAgICByZXR1cm4gMAogICAgICBmaQoKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUF9QUk9YWV9VUkx9IiAiJHtzdXBwb3J0ZWRfcHJvdG9jb2xzW0BdfSIKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUFNfUFJPWFlfVVJMfSIgIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iCgogICAgICBpZiAhIGdyZXAgLXEgJ2h0dHBfcHJveHknIC9ldGMvZW52aXJvbm1lbnQ7IHRoZW4KICAgICAgICBlY2hvICJleHBvcnQgaHR0cF9wcm94eT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBodHRwc19wcm94eT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgSFRUUF9QUk9YWT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBIVFRQU19QUk9YWT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgbm9fcHJveHk9bG9jYWxob3N0LDEyNy4wLjAuMSIgPj4gL2V0Yy9lbnZpcm9ubWVudAogICAgICAgIHNvdXJjZSAvZXRjL2Vudmlyb25tZW50CiAgICAgIGZpCiAgICAgIAogICAgICAjIENvbmZpZ3VyZSBEb2NrZXIgdG8gdXNlIGEgcHJveHkKICAgICAgbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCiAgICAgIGVjaG8gIltTZXJ2aWNlXQogICAgICBFbnZpcm9ubWVudD1cIkhUVFBfUFJPWFk9JHtIVFRQX1BST1hZX1VSTH1cIgogICAgICBFbnZpcm9ubWVudD1cIkhUVFBTX1BST1hZPSR7SFRUUFNfUFJPWFlfVVJMfVwiCiAgICAgIEVudmlyb25tZW50PVwiTk9fUFJPWFk9bG9jYWxob3N0LDEyNy4wLjAuMVwiIiA+IC9ldGMvc3lzdGVtZC9zeXN0ZW0vZG9ja2VyLnNlcnZpY2UuZC9wcm94eS5jb25mCiAgICAgIHN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCiAgICAgIHN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgoKICAgICAgZWNobyAiSW5mbzogZG9ja2VyIGFuZCBzeXN0ZW0gZW52aXJvbm1lbnQgYXJlIG5vdyBjb25maWd1cmVkIHRvIHVzZSB0aGUgcHJveHkgc2V0dGluZ3MiCiAgICB9 che corrisponde allo script seguente in formato testo normale: #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all -p 8888:8888 $REGISTRY_URI_PATH/nvidia/tensorflow:23.10-tf2-py3 /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Immagine one-liner. Codificarlo nel formato base64. docker run -d -p 8888:8888 nvcr.io/nvidia/tensorflow:ngc_image_tag /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace Ad esempio, per tensorflow:23.10-tf2-py3, specificare lo script seguente in formato base64: ZG9ja2VyIHJ1biAtZCAtcCA4ODg4Ojg4ODggbnZjci5pby9udmlkaWEvdGVuc29yZmxvdzoyMy4xMC10ZjItcHkzIC91c3IvbG9jYWwvYmluL2p1cHl0ZXIgbGFiIC0tYWxsb3ctcm9vdCAtLWlwPSogLS1wb3J0PTg4ODggLS1uby1icm93c2VyIC0tTm90ZWJvb2tBcHAudG9rZW49JycgLS1Ob3RlYm9va0FwcC5hbGxvd19vcmlnaW49JyonIC0tbm90ZWJvb2stZGlyPS93b3Jrc3BhY2U= che corrisponde allo script seguente in formato testo normale: docker run -d -p 8888:8888 nvcr.io/nvidia/tensorflow:23.10-tf2-py3 /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace Immettere le proprietà di installazione del driver guest vGPU, ad esempio vgpu-license e nvidia-portal-api-key. Specificare i valori per le proprietà necessarie per un ambiente disconnesso in base alle esigenze. Vedere Proprietà OVF delle macchine virtuali di deep learning.
Output	Registri di installazione per il driver guest della vGPU in /var/log/vgpu-install.log. Per verificare che il driver guest della vGPU sia installato, accedere alla macchina virtuale tramite SSH ed eseguire il comando nvidia-smi. Registri dello script cloud-init in /var/log/dl.log. Container TensorFlow. Per verificare che il container TensorFlow sia in esecuzione, eseguire i comandi sudo docker ps -a e sudo docker logs container_id. Istanza di JupyterLab a cui è possibile accedere in http://dl_vm_ip:8888. Nel terminale di JupyterLab, verificare che nel notebook siano disponibili le funzionalità seguenti: Per verificare che JupyterLab possa accedere alla risorsa vGPU, eseguire nvidia-smi. Per verificare che i pacchetti relativi a TensorFlow siano installati, eseguire pip show.

DCGM Exporter

È possibile utilizzare una macchina virtuale di deep learning con Data Center GPU Manager (DCGM) Exporter per monitorare l'integrità delle GPU e ottenerne le metriche utilizzate da un carico di lavoro DL, tramite NVIDIA DCGM, Prometheus e Grafana.

Vedere la pagina DCGM Exporter.

In una macchina virtuale di deep learning eseguire il container DCGM Exporter insieme a un carico di lavoro DL che esegue le operazioni di AI. Dopo l'avvio di Deep Learning VM, DCGM Exporter è pronto a raccogliere le metriche di vGPU ed esportare i dati in un'altra applicazione per ulteriore monitoraggio e visualizzazione. È possibile eseguire il carico di lavoro DL monitorato come parte del processo cloud-init o dalla riga di comando dopo l'avvio della macchina virtuale.

Tabella 4. Immagine del container DCGM Exporter

Componente	Descrizione
Immagine del container	nvcr.io/nvidia/k8s/dcgm-exporter:ngc_image_tag Ad esempio: nvcr.io/nvidia/k8s/dcgm-exporter:3.2.5-3.1.8-ubuntu22.04 Per informazioni sulle immagini di container di DCGM Exporter supportate per le macchine virtuali di deep learning, vedere Note di rilascio di VMware Deep Learning VM.
Input necessari	Per distribuire un carico di lavoro DCGM Exporter, è necessario impostare le proprietà OVF per la macchina virtuale di deep learning nel modo seguente: Utilizzare una delle seguenti proprietà specifiche dell'immagine di DCGM Exporter. Script cloud-init. Codificarlo nel formato base64. #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all --cap-add SYS_ADMIN --rm -p 9400:9400 $REGISTRY_URI_PATH/nvidia/k8s/dcgm-exporter:ngc_image_tag - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Ad esempio, per una macchina virtuale di deep learning in cui è preinstallata un'istanza di DCGM Exporter dcgm-exporter:3.2.5-3.1.8-ubuntu22.04, specificare lo script seguente in formato base64 I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBzZXQgLWV1CiAgICBzb3VyY2UgL29wdC9kbHZtL3V0aWxzLnNoCiAgICB0cmFwICdlcnJvcl9leGl0ICJVbmV4cGVjdGVkIGVycm9yIG9jY3VycyBhdCBkbCB3b3JrbG9hZCInIEVSUgogICAgc2V0X3Byb3h5ICJodHRwIiAiaHR0cHMiICJzb2NrczUiCiAgICAKICAgIERFRkFVTFRfUkVHX1VSST0ibnZjci5pbyIKICAgIFJFR0lTVFJZX1VSSV9QQVRIPSQoZ3JlcCByZWdpc3RyeS11cmkgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQoKICAgIGlmIFtbIC16ICIkUkVHSVNUUllfVVJJX1BBVEgiIF1dOyB0aGVuCiAgICAgICMgSWYgUkVHSVNUUllfVVJJX1BBVEggaXMgbnVsbCBvciBlbXB0eSwgdXNlIHRoZSBkZWZhdWx0IHZhbHVlCiAgICAgIFJFR0lTVFJZX1VSSV9QQVRIPSRERUZBVUxUX1JFR19VUkkKICAgICAgZWNobyAiUkVHSVNUUllfVVJJX1BBVEggd2FzIGVtcHR5LiBVc2luZyBkZWZhdWx0OiAkUkVHSVNUUllfVVJJX1BBVEgiCiAgICBmaQogICAgCiAgICAjIElmIFJFR0lTVFJZX1VSSV9QQVRIIGNvbnRhaW5zICcvJywgZXh0cmFjdCB0aGUgVVJJIHBhcnQKICAgIGlmIFtbICRSRUdJU1RSWV9VUklfUEFUSCA9PSAqIi8iKiBdXTsgdGhlbgogICAgICBSRUdJU1RSWV9VUkk9JChlY2hvICIkUkVHSVNUUllfVVJJX1BBVEgiIHwgY3V0IC1kJy8nIC1mMSkKICAgIGVsc2UKICAgICAgUkVHSVNUUllfVVJJPSRSRUdJU1RSWV9VUklfUEFUSAogICAgZmkKICAKICAgIFJFR0lTVFJZX1VTRVJOQU1FPSQoZ3JlcCByZWdpc3RyeS11c2VyIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKICAgIFJFR0lTVFJZX1BBU1NXT1JEPSQoZ3JlcCByZWdpc3RyeS1wYXNzd2QgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgaWYgW1sgLW4gIiRSRUdJU1RSWV9VU0VSTkFNRSIgJiYgLW4gIiRSRUdJU1RSWV9QQVNTV09SRCIgXV07IHRoZW4KICAgICAgZG9ja2VyIGxvZ2luIC11ICRSRUdJU1RSWV9VU0VSTkFNRSAtcCAkUkVHSVNUUllfUEFTU1dPUkQgJFJFR0lTVFJZX1VSSQogICAgZWxzZQogICAgICBlY2hvICJXYXJuaW5nOiB0aGUgcmVnaXN0cnkncyB1c2VybmFtZSBhbmQgcGFzc3dvcmQgYXJlIGludmFsaWQsIFNraXBwaW5nIERvY2tlciBsb2dpbi4iCiAgICBmaQoKICAgIGRvY2tlciBydW4gLWQgLS1ncHVzIGFsbCAtLWNhcC1hZGQgU1lTX0FETUlOIC0tcm0gLXAgOTQwMDo5NDAwICRSRUdJU1RSWV9VUklfUEFUSC9udmlkaWEvazhzL2RjZ20tZXhwb3J0ZXI6My4yLjUtMy4xLjgtdWJ1bnR1MjIuMDQKCi0gcGF0aDogL29wdC9kbHZtL3V0aWxzLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBlcnJvcl9leGl0KCkgewogICAgICBlY2hvICJFcnJvcjogJDEiID4mMgogICAgICB2bXRvb2xzZCAtLWNtZCAiaW5mby1zZXQgZ3Vlc3RpbmZvLnZtc2VydmljZS5ib290c3RyYXAuY29uZGl0aW9uIGZhbHNlLCBETFdvcmtsb2FkRmFpbHVyZSwgJDEiCiAgICAgIGV4aXQgMQogICAgfQoKICAgIGNoZWNrX3Byb3RvY29sKCkgewogICAgICBsb2NhbCBwcm94eV91cmw9JDEKICAgICAgc2hpZnQKICAgICAgbG9jYWwgc3VwcG9ydGVkX3Byb3RvY29scz0oIiRAIikKICAgICAgaWYgW1sgLW4gIiR7cHJveHlfdXJsfSIgXV07IHRoZW4KICAgICAgICBsb2NhbCBwcm90b2NvbD0kKGVjaG8gIiR7cHJveHlfdXJsfSIgfCBhd2sgLUYgJzovLycgJ3tpZiAoTkYgPiAxKSBwcmludCAkMTsgZWxzZSBwcmludCAiIn0nKQogICAgICAgIGlmIFsgLXogIiRwcm90b2NvbCIgXTsgdGhlbgogICAgICAgICAgZWNobyAiTm8gc3BlY2lmaWMgcHJvdG9jb2wgcHJvdmlkZWQuIFNraXBwaW5nIHByb3RvY29sIGNoZWNrLiIKICAgICAgICAgIHJldHVybiAwCiAgICAgICAgZmkKICAgICAgICBsb2NhbCBwcm90b2NvbF9pbmNsdWRlZD1mYWxzZQogICAgICAgIGZvciB2YXIgaW4gIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iOyBkbwogICAgICAgICAgaWYgW1sgIiR7cHJvdG9jb2x9IiA9PSAiJHt2YXJ9IiBdXTsgdGhlbgogICAgICAgICAgICBwcm90b2NvbF9pbmNsdWRlZD10cnVlCiAgICAgICAgICAgIGJyZWFrCiAgICAgICAgICBmaQogICAgICAgIGRvbmUKICAgICAgICBpZiBbWyAiJHtwcm90b2NvbF9pbmNsdWRlZH0iID09IGZhbHNlIF1dOyB0aGVuCiAgICAgICAgICBlcnJvcl9leGl0ICJVbnN1cHBvcnRlZCBwcm90b2NvbDogJHtwcm90b2NvbH0uIFN1cHBvcnRlZCBwcm90b2NvbHMgYXJlOiAke3N1cHBvcnRlZF9wcm90b2NvbHNbKl19IgogICAgICAgIGZpCiAgICAgIGZpCiAgICB9CgogICAgIyAkQDogbGlzdCBvZiBzdXBwb3J0ZWQgcHJvdG9jb2xzCiAgICBzZXRfcHJveHkoKSB7CiAgICAgIGxvY2FsIHN1cHBvcnRlZF9wcm90b2NvbHM9KCIkQCIpCgogICAgICBDT05GSUdfSlNPTl9CQVNFNjQ9JChncmVwICdjb25maWctanNvbicgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgICBDT05GSUdfSlNPTj0kKGVjaG8gJHtDT05GSUdfSlNPTl9CQVNFNjR9IHwgYmFzZTY0IC0tZGVjb2RlKQoKICAgICAgSFRUUF9QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBfcHJveHkgLy8gZW1wdHknKQogICAgICBIVFRQU19QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBzX3Byb3h5IC8vIGVtcHR5JykKICAgICAgaWYgW1sgJD8gLW5lIDAgfHwgKC16ICIke0hUVFBfUFJPWFlfVVJMfSIgJiYgLXogIiR7SFRUUFNfUFJPWFlfVVJMfSIpIF1dOyB0aGVuCiAgICAgICAgZWNobyAiSW5mbzogVGhlIGNvbmZpZy1qc29uIHdhcyBwYXJzZWQsIGJ1dCBubyBwcm94eSBzZXR0aW5ncyB3ZXJlIGZvdW5kLiIKICAgICAgICByZXR1cm4gMAogICAgICBmaQoKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUF9QUk9YWV9VUkx9IiAiJHtzdXBwb3J0ZWRfcHJvdG9jb2xzW0BdfSIKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUFNfUFJPWFlfVVJMfSIgIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iCgogICAgICBpZiAhIGdyZXAgLXEgJ2h0dHBfcHJveHknIC9ldGMvZW52aXJvbm1lbnQ7IHRoZW4KICAgICAgICBlY2hvICJleHBvcnQgaHR0cF9wcm94eT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBodHRwc19wcm94eT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgSFRUUF9QUk9YWT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBIVFRQU19QUk9YWT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgbm9fcHJveHk9bG9jYWxob3N0LDEyNy4wLjAuMSIgPj4gL2V0Yy9lbnZpcm9ubWVudAogICAgICAgIHNvdXJjZSAvZXRjL2Vudmlyb25tZW50CiAgICAgIGZpCiAgICAgIAogICAgICAjIENvbmZpZ3VyZSBEb2NrZXIgdG8gdXNlIGEgcHJveHkKICAgICAgbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCiAgICAgIGVjaG8gIltTZXJ2aWNlXQogICAgICBFbnZpcm9ubWVudD1cIkhUVFBfUFJPWFk9JHtIVFRQX1BST1hZX1VSTH1cIgogICAgICBFbnZpcm9ubWVudD1cIkhUVFBTX1BST1hZPSR7SFRUUFNfUFJPWFlfVVJMfVwiCiAgICAgIEVudmlyb25tZW50PVwiTk9fUFJPWFk9bG9jYWxob3N0LDEyNy4wLjAuMVwiIiA+IC9ldGMvc3lzdGVtZC9zeXN0ZW0vZG9ja2VyLnNlcnZpY2UuZC9wcm94eS5jb25mCiAgICAgIHN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCiAgICAgIHN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgoKICAgICAgZWNobyAiSW5mbzogZG9ja2VyIGFuZCBzeXN0ZW0gZW52aXJvbm1lbnQgYXJlIG5vdyBjb25maWd1cmVkIHRvIHVzZSB0aGUgcHJveHkgc2V0dGluZ3MiCiAgICB9 che corrisponde allo script seguente in formato testo normale: #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all --cap-add SYS_ADMIN --rm -p 9400:9400 $REGISTRY_URI_PATH/nvidia/k8s/dcgm-exporter:3.2.5-3.1.8-ubuntu22.04 - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Nota: Nello script cloud-init è inoltre possibile aggiungere le istruzioni per l'esecuzione del carico di lavoro DL di cui si desidera misurare le prestazioni della GPU con DCGM Exporter. Immagine one-liner. Codificarlo nel formato base64. docker run -d --gpus all --cap-add SYS_ADMIN --rm -p 9400:9400 nvcr.io/nvidia/k8s/dcgm-exporter:ngc_image_tag-ubuntu22.04 Ad esempio, per dcgm-exporter:3.2.5-3.1.8-ubuntu22.04, specificare lo script seguente in formato base64: ZG9ja2VyIHJ1biAtZCAtLWdwdXMgYWxsIC0tY2FwLWFkZCBTWVNfQURNSU4gLS1ybSAtcCA5NDAwOjk0MDAgbnZjci5pby9udmlkaWEvazhzL2RjZ20tZXhwb3J0ZXI6My4yLjUtMy4xLjgtdWJ1bnR1MjIuMDQ= che corrisponde allo script seguente in formato testo normale: docker run -d --gpus all --cap-add SYS_ADMIN --rm -p 9400:9400 nvcr.io/nvidia/k8s/dcgm-exporter:3.2.5-3.1.8-ubuntu22.04 Immettere le proprietà di installazione del driver guest vGPU, ad esempio vgpu-license e nvidia-portal-api-key. Specificare i valori per le proprietà necessarie per un ambiente disconnesso in base alle esigenze. Vedere Proprietà OVF delle macchine virtuali di deep learning.
Output	Registri di installazione per il driver guest della vGPU in /var/log/vgpu-install.log. Per verificare che il driver guest della vGPU sia installato, accedere alla macchina virtuale tramite SSH ed eseguire il comando nvidia-smi. Registri dello script cloud-init in /var/log/dl.log. DCGM Exporter a cui è possibile accedere all'indirizzo http://dl_vm_ip:9400. Nella macchina virtuale di deep learning, eseguire quindi un carico di lavoro DL e visualizzare i dati in un'altra macchina virtuale utilizzando Prometheus all'indirizzo http://visualization_vm_ip:9090 e Grafana all'indirizzo http://visualization_vm_ip:3000.

Esecuzione di un carico di lavoro DL nella macchina virtuale di deep leaning

Eseguire il carico di lavoro DL per cui si desidera raccogliere le metriche vGPU ed esportare i dati in un'altra applicazione per ulteriori informazioni di monitoraggio e visualizzazione.

1. Accedere alla macchina virtuale di deep learning come vmware tramite SSH.
2. Aggiungere l'account utente vmware al gruppo docker eseguendo il comando seguente.

   sudo usermod -aG docker ${USER}

3. Eseguire il container per il carico di lavoro DL, estraendolo dal catalogo NVIDIA NGC o da un registro di container locale.

   Ad esempio, per eseguire il comando seguente per l'esecuzione dell'immagine tensorflow:23.10-tf2-py3 da NVIDIA NGC:

   docker run -d -p 8888:8888 nvcr.io/nvidia/tensorflow:23.10-tf2-py3 /usr/local/bin/jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' --notebook-dir=/workspace

4. Iniziare a utilizzare il carico di lavoro DL per lo sviluppo di AI.

Installazione di Prometheus e Grafana

È possibile visualizzare e monitorare le metriche della vGPU dalla macchina virtuale di DCGM Exporter in una macchina virtuale che esegue Prometheus e Grafana.

1. Creare una macchina virtuale di visualizzazione in cui è installato Docker Community Engine.
2. Connettersi alla macchina virtuale tramite SSH e creare un file YAML per Prometheus.

   $ cat > prometheus.yml << EOF
   global:
     scrape_interval: 15s
     external_labels:
       monitor: 'codelab-monitor'
   scrape_configs:
     - job_name: 'dcgm'
       scrape_interval: 5s
       metrics_path: /metrics
       static_configs:
         - targets: [dl_vm_with_dcgm_exporter_ip:9400']
   EOF
   

3. Creare un percorso dati.

   $ mkdir grafana_data prometheus_data && chmod 777 grafana_data prometheus_data
   

4. Creare un file di composizione Docker per installare Prometheus e Grafana.

   $ cat > compose.yaml << EOF
   services:
     prometheus:
       image: prom/prometheus:v2.47.2
       container_name: "prometheus0"
       restart: always
       ports:
         - "9090:9090"
       volumes:
         - "./prometheus.yml:/etc/prometheus/prometheus.yml"
         - "./prometheus_data:/prometheus"
     grafana:
       image: grafana/grafana:10.2.0-ubuntu
       container_name: "grafana0"
       ports:
         - "3000:3000"
       restart: always
       volumes:
         - "./grafana_data:/var/lib/grafana"
   EOF
   

5. Avviare i container di Prometheus e Grafana.

   $ sudo docker compose up -d        
   

Visualizzazione delle metriche della vGPU in Prometheus

È possibile accedere a Prometheus all'indirizzo http://visualization-vm-ip:9090. È possibile visualizzare le seguenti informazioni sulla vGPU nell'interfaccia utente di Prometheus:

Informazioni	Sezione dell'interfaccia utente
Metriche della vGPU non elaborate dalla macchina virtuale di deep learning	Stato > Destinazione Per visualizzare le metriche della vGPU non elaborate dalla macchina virtuale di deep learning, fare clic sulla voce dell'endpoint.
Espressioni del grafico	Nella barra di navigazione principale, fare clic sulla scheda Grafico. Immettere un'espressione e fare clic su Esegui

Per ulteriori informazioni sull'utilizzo di Prometheus, vedere la documentazione di Prometheus.

Visualizzazione delle metriche in Grafana

Impostare Prometheus come origine dati per Grafana e visualizzare le metriche della vGPU dalla macchina virtuale di deep learning in un dashboard.

1. Accedere a Grafana all'indirizzo http://visualization-vm-ip:3000 utilizzando il nome utente predefinito admin e la password admin.
2. Aggiungere Prometheus come prima origine dati connettendosi a visualization-vm-ip nella porta 9090.
3. Creare un dashboard con le metriche della vGPU.

Per ulteriori informazioni sulla configurazione di un dashboard utilizzando un'origine dati Prometheus, vedere la documentazione di Grafana.

Triton Inference Server

È possibile utilizzare una macchina virtuale di deep learning con Triton Inference Server per caricare un repository di modelli e ricevere richieste di inferenza.

Vedere la pagina Triton Inference Server.

Tabella 5. Immagine del container Triton Inference Server

Componente	Descrizione
Immagine del container	nvcr.io/nvidia/tritonserver:ngc_image_tag Ad esempio: nvcr.io/nvidia/tritonserver:23.10-py3 Per informazioni sulle immagini dei container Triton Inference Server supportate per le macchine virtuali di deep learning, vedere Note di rilascio di VMware Deep Learning VM.
Input necessari	Per distribuire un carico di lavoro Triton Inference Server, è necessario impostare le proprietà OVF per la macchina virtuale di deep learning nel modo seguente: Utilizzare una delle proprietà seguenti specifiche per l'immagine di Triton Inference Server. Script cloud-init. Codificarlo nel formato base64. #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v /home/vmware/model_repository:/models $REGISTRY_URI_PATH/nvidia/tritonserver:ngc_image_tag tritonserver --model-repository=/models --model-control-mode=poll - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Ad esempio, per tritonserver:23.10-py3, specificare lo script seguente in formato base64: I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBzZXQgLWV1CiAgICBzb3VyY2UgL29wdC9kbHZtL3V0aWxzLnNoCiAgICB0cmFwICdlcnJvcl9leGl0ICJVbmV4cGVjdGVkIGVycm9yIG9jY3VycyBhdCBkbCB3b3JrbG9hZCInIEVSUgogICAgc2V0X3Byb3h5ICJodHRwIiAiaHR0cHMiICJzb2NrczUiCgogICAgREVGQVVMVF9SRUdfVVJJPSJudmNyLmlvIgogICAgUkVHSVNUUllfVVJJX1BBVEg9JChncmVwIHJlZ2lzdHJ5LXVyaSAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpCgogICAgaWYgW1sgLXogIiRSRUdJU1RSWV9VUklfUEFUSCIgXV07IHRoZW4KICAgICAgIyBJZiBSRUdJU1RSWV9VUklfUEFUSCBpcyBudWxsIG9yIGVtcHR5LCB1c2UgdGhlIGRlZmF1bHQgdmFsdWUKICAgICAgUkVHSVNUUllfVVJJX1BBVEg9JERFRkFVTFRfUkVHX1VSSQogICAgICBlY2hvICJSRUdJU1RSWV9VUklfUEFUSCB3YXMgZW1wdHkuIFVzaW5nIGRlZmF1bHQ6ICRSRUdJU1RSWV9VUklfUEFUSCIKICAgIGZpCiAgICAKICAgICMgSWYgUkVHSVNUUllfVVJJX1BBVEggY29udGFpbnMgJy8nLCBleHRyYWN0IHRoZSBVUkkgcGFydAogICAgaWYgW1sgJFJFR0lTVFJZX1VSSV9QQVRIID09ICoiLyIqIF1dOyB0aGVuCiAgICAgIFJFR0lTVFJZX1VSST0kKGVjaG8gIiRSRUdJU1RSWV9VUklfUEFUSCIgfCBjdXQgLWQnLycgLWYxKQogICAgZWxzZQogICAgICBSRUdJU1RSWV9VUkk9JFJFR0lTVFJZX1VSSV9QQVRICiAgICBmaQogIAogICAgUkVHSVNUUllfVVNFUk5BTUU9JChncmVwIHJlZ2lzdHJ5LXVzZXIgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgUkVHSVNUUllfUEFTU1dPUkQ9JChncmVwIHJlZ2lzdHJ5LXBhc3N3ZCAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpCiAgICBpZiBbWyAtbiAiJFJFR0lTVFJZX1VTRVJOQU1FIiAmJiAtbiAiJFJFR0lTVFJZX1BBU1NXT1JEIiBdXTsgdGhlbgogICAgICBkb2NrZXIgbG9naW4gLXUgJFJFR0lTVFJZX1VTRVJOQU1FIC1wICRSRUdJU1RSWV9QQVNTV09SRCAkUkVHSVNUUllfVVJJCiAgICBlbHNlCiAgICAgIGVjaG8gIldhcm5pbmc6IHRoZSByZWdpc3RyeSdzIHVzZXJuYW1lIGFuZCBwYXNzd29yZCBhcmUgaW52YWxpZCwgU2tpcHBpbmcgRG9ja2VyIGxvZ2luLiIKICAgIGZpCgogICAgZG9ja2VyIHJ1biAtZCAtLWdwdXMgYWxsIC0tcm0gLXAgODAwMDo4MDAwIC1wIDgwMDE6ODAwMSAtcCA4MDAyOjgwMDIgLXYgL2hvbWUvdm13YXJlL21vZGVsX3JlcG9zaXRvcnk6L21vZGVscyAkUkVHSVNUUllfVVJJX1BBVEgvbnZpZGlhL3RyaXRvbnNlcnZlcjoyMy4xMC1weTMgdHJpdG9uc2VydmVyIC0tbW9kZWwtcmVwb3NpdG9yeT0vbW9kZWxzIC0tbW9kZWwtY29udHJvbC1tb2RlPXBvbGwKCi0gcGF0aDogL29wdC9kbHZtL3V0aWxzLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBlcnJvcl9leGl0KCkgewogICAgICBlY2hvICJFcnJvcjogJDEiID4mMgogICAgICB2bXRvb2xzZCAtLWNtZCAiaW5mby1zZXQgZ3Vlc3RpbmZvLnZtc2VydmljZS5ib290c3RyYXAuY29uZGl0aW9uIGZhbHNlLCBETFdvcmtsb2FkRmFpbHVyZSwgJDEiCiAgICAgIGV4aXQgMQogICAgfQoKICAgIGNoZWNrX3Byb3RvY29sKCkgewogICAgICBsb2NhbCBwcm94eV91cmw9JDEKICAgICAgc2hpZnQKICAgICAgbG9jYWwgc3VwcG9ydGVkX3Byb3RvY29scz0oIiRAIikKICAgICAgaWYgW1sgLW4gIiR7cHJveHlfdXJsfSIgXV07IHRoZW4KICAgICAgICBsb2NhbCBwcm90b2NvbD0kKGVjaG8gIiR7cHJveHlfdXJsfSIgfCBhd2sgLUYgJzovLycgJ3tpZiAoTkYgPiAxKSBwcmludCAkMTsgZWxzZSBwcmludCAiIn0nKQogICAgICAgIGlmIFsgLXogIiRwcm90b2NvbCIgXTsgdGhlbgogICAgICAgICAgZWNobyAiTm8gc3BlY2lmaWMgcHJvdG9jb2wgcHJvdmlkZWQuIFNraXBwaW5nIHByb3RvY29sIGNoZWNrLiIKICAgICAgICAgIHJldHVybiAwCiAgICAgICAgZmkKICAgICAgICBsb2NhbCBwcm90b2NvbF9pbmNsdWRlZD1mYWxzZQogICAgICAgIGZvciB2YXIgaW4gIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iOyBkbwogICAgICAgICAgaWYgW1sgIiR7cHJvdG9jb2x9IiA9PSAiJHt2YXJ9IiBdXTsgdGhlbgogICAgICAgICAgICBwcm90b2NvbF9pbmNsdWRlZD10cnVlCiAgICAgICAgICAgIGJyZWFrCiAgICAgICAgICBmaQogICAgICAgIGRvbmUKICAgICAgICBpZiBbWyAiJHtwcm90b2NvbF9pbmNsdWRlZH0iID09IGZhbHNlIF1dOyB0aGVuCiAgICAgICAgICBlcnJvcl9leGl0ICJVbnN1cHBvcnRlZCBwcm90b2NvbDogJHtwcm90b2NvbH0uIFN1cHBvcnRlZCBwcm90b2NvbHMgYXJlOiAke3N1cHBvcnRlZF9wcm90b2NvbHNbKl19IgogICAgICAgIGZpCiAgICAgIGZpCiAgICB9CgogICAgIyAkQDogbGlzdCBvZiBzdXBwb3J0ZWQgcHJvdG9jb2xzCiAgICBzZXRfcHJveHkoKSB7CiAgICAgIGxvY2FsIHN1cHBvcnRlZF9wcm90b2NvbHM9KCIkQCIpCgogICAgICBDT05GSUdfSlNPTl9CQVNFNjQ9JChncmVwICdjb25maWctanNvbicgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgICBDT05GSUdfSlNPTj0kKGVjaG8gJHtDT05GSUdfSlNPTl9CQVNFNjR9IHwgYmFzZTY0IC0tZGVjb2RlKQoKICAgICAgSFRUUF9QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBfcHJveHkgLy8gZW1wdHknKQogICAgICBIVFRQU19QUk9YWV9VUkw9JChlY2hvICIke0NPTkZJR19KU09OfSIgfCBqcSAtciAnLmh0dHBzX3Byb3h5IC8vIGVtcHR5JykKICAgICAgaWYgW1sgJD8gLW5lIDAgfHwgKC16ICIke0hUVFBfUFJPWFlfVVJMfSIgJiYgLXogIiR7SFRUUFNfUFJPWFlfVVJMfSIpIF1dOyB0aGVuCiAgICAgICAgZWNobyAiSW5mbzogVGhlIGNvbmZpZy1qc29uIHdhcyBwYXJzZWQsIGJ1dCBubyBwcm94eSBzZXR0aW5ncyB3ZXJlIGZvdW5kLiIKICAgICAgICByZXR1cm4gMAogICAgICBmaQoKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUF9QUk9YWV9VUkx9IiAiJHtzdXBwb3J0ZWRfcHJvdG9jb2xzW0BdfSIKICAgICAgY2hlY2tfcHJvdG9jb2wgIiR7SFRUUFNfUFJPWFlfVVJMfSIgIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iCgogICAgICBpZiAhIGdyZXAgLXEgJ2h0dHBfcHJveHknIC9ldGMvZW52aXJvbm1lbnQ7IHRoZW4KICAgICAgICBlY2hvICJleHBvcnQgaHR0cF9wcm94eT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBodHRwc19wcm94eT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgSFRUUF9QUk9YWT0ke0hUVFBfUFJPWFlfVVJMfQogICAgICAgIGV4cG9ydCBIVFRQU19QUk9YWT0ke0hUVFBTX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgbm9fcHJveHk9bG9jYWxob3N0LDEyNy4wLjAuMSIgPj4gL2V0Yy9lbnZpcm9ubWVudAogICAgICAgIHNvdXJjZSAvZXRjL2Vudmlyb25tZW50CiAgICAgIGZpCiAgICAgIAogICAgICAjIENvbmZpZ3VyZSBEb2NrZXIgdG8gdXNlIGEgcHJveHkKICAgICAgbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCiAgICAgIGVjaG8gIltTZXJ2aWNlXQogICAgICBFbnZpcm9ubWVudD1cIkhUVFBfUFJPWFk9JHtIVFRQX1BST1hZX1VSTH1cIgogICAgICBFbnZpcm9ubWVudD1cIkhUVFBTX1BST1hZPSR7SFRUUFNfUFJPWFlfVVJMfVwiCiAgICAgIEVudmlyb25tZW50PVwiTk9fUFJPWFk9bG9jYWxob3N0LDEyNy4wLjAuMVwiIiA+IC9ldGMvc3lzdGVtZC9zeXN0ZW0vZG9ja2VyLnNlcnZpY2UuZC9wcm94eS5jb25mCiAgICAgIHN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCiAgICAgIHN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgoKICAgICAgZWNobyAiSW5mbzogZG9ja2VyIGFuZCBzeXN0ZW0gZW52aXJvbm1lbnQgYXJlIG5vdyBjb25maWd1cmVkIHRvIHVzZSB0aGUgcHJveHkgc2V0dGluZ3MiCiAgICB9 che corrisponde allo script seguente in formato testo normale: #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" "socks5" DEFAULT_REG_URI="nvcr.io" REGISTRY_URI_PATH=$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -z "$REGISTRY_URI_PATH" ]]; then # If REGISTRY_URI_PATH is null or empty, use the default value REGISTRY_URI_PATH=$DEFAULT_REG_URI echo "REGISTRY_URI_PATH was empty. Using default: $REGISTRY_URI_PATH" fi # If REGISTRY_URI_PATH contains '/', extract the URI part if [[ $REGISTRY_URI_PATH == *"/"* ]]; then REGISTRY_URI=$(echo "$REGISTRY_URI_PATH" | cut -d'/' -f1) else REGISTRY_URI=$REGISTRY_URI_PATH fi REGISTRY_USERNAME=$(grep registry-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') REGISTRY_PASSWORD=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "$REGISTRY_USERNAME" && -n "$REGISTRY_PASSWORD" ]]; then docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD $REGISTRY_URI else echo "Warning: the registry's username and password are invalid, Skipping Docker login." fi docker run -d --gpus all --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v /home/vmware/model_repository:/models $REGISTRY_URI_PATH/nvidia/tritonserver:23.10-py3 tritonserver --model-repository=/models --model-control-mode=poll - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Immagine one-liner codificata nel formato base64 docker run -d --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/vmware/model_repository:/models nvcr.io/nvidia/tritonserver:ngc_image_tag tritonserver --model-repository=/models --model-control-mode=poll Ad esempio, per tritonserver:23.10-py3, specificare lo script seguente in formato base64: ZG9ja2VyIHJ1biAtZCAtLWdwdXMgYWxsIC0tcm0gLXA4MDAwOjgwMDAgLXA4MDAxOjgwMDEgLXA4MDAyOjgwMDIgLXYgL2hvbWUvdm13YXJlL21vZGVsX3JlcG9zaXRvcnk6L21vZGVscyBudmNyLmlvL252aWRpYS90cml0b25zZXJ2ZXI6MjMuMTAtcHkzIHRyaXRvbnNlcnZlciAtLW1vZGVsLXJlcG9zaXRvcnk9L21vZGVscyAtLW1vZGVsLWNvbnRyb2wtbW9kZT1wb2xs che corrisponde allo script seguente in formato testo normale: docker run -d --gpus all --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /home/vmware/model_repository:/models nvcr.io/nvidia/tritonserver:23.10-py3 tritonserver --model-repository=/models --model-control-mode=poll Immettere le proprietà di installazione del driver guest vGPU, ad esempio vgpu-license e nvidia-portal-api-key. Specificare i valori per le proprietà necessarie per un ambiente disconnesso in base alle esigenze. Vedere Proprietà OVF delle macchine virtuali di deep learning.
Output	Registri di installazione per il driver guest della vGPU in /var/log/vgpu-install.log. Per verificare che il driver guest della vGPU sia installato, accedere alla macchina virtuale tramite SSH ed eseguire il comando nvidia-smi. Registri dello script cloud-init in /var/log/dl.log. Container Triton Inference Server. Per verificare che il container Triton Inference Server sia in esecuzione, eseguire i comandi sudo docker ps -a e sudo docker logs container_id. Il repository di modelli per Triton Inference Server è in /home/vmware/model_repository. Inizialmente, il repository di modelli è vuoto e il registro iniziale dell'istanza di Triton Inference Server indica che non è stato caricato alcun modello.

Creazione di un repository di modelli

Per caricare il modello per l'inferenza del modello, eseguire i passaggi seguenti:

1. Creare il repository di modelli per il proprio modello.

   Vedere la documentazione relativa al repository di modelli di NVIDIA Triton Inference Server.

2. Copiare il repository di modelli in /home/vmware/model_repository in modo che Triton Inference Server possa caricarlo.

   sudo cp -r path_to_your_created_model_repository/* /home/vmware/model_repository/
   

Invio di richieste di inferenza del modello

1. Verificare che Triton Inference Server sia integro e che i modelli siano pronti eseguendo questo comando nella console della macchina virtuale di deep learning.

   curl -v localhost:8000/v2/simple_sequence

2. Inviare una richiesta al modello eseguendo questo comando nella macchina virtuale di deep learning.

    curl -v localhost:8000/v2/models/simple_sequence

Per ulteriori informazioni sull'utilizzo di Triton Inference Server, vedere la documentazione relativa al repository di modelli di NVIDIA Triton Inference Server.

NVIDIA RAG

È possibile utilizzare una macchina virtuale di deep learning per creare soluzioni RAG (Retrieval Augmented Generation) con un modello Llama2.

Vedere la documentazione NVIDIA RAG Applications Docker Compose (richiede autorizzazioni dell'account specifiche).

Tabella 6. Immagine del container NVIDIA RAG

Componente	Descrizione
Immagini e modelli di container	rag-app-text-chatbot.yaml nella pipeline di NVIDIA RAG di esempio. Per informazioni sulle applicazioni container NVIDIA RAG supportate per le macchine virtuali di deep learning, vedere Note di rilascio di VMware Deep Learning VM.
Input necessari	Per distribuire un carico di lavoro NVIDIA RAG, è necessario impostare le proprietà OVF per la macchina virtuale di deep learning nel modo seguente: Immettere uno script cloud-init. Codificarlo nel formato base64. Ad esempio, per la versione 24.03 di NVIDIA RAG, specificare lo script seguente: I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBzZXQgLWV1CiAgICBzb3VyY2UgL29wdC9kbHZtL3V0aWxzLnNoCiAgICB0cmFwICdlcnJvcl9leGl0ICJVbmV4cGVjdGVkIGVycm9yIG9jY3VycyBhdCBkbCB3b3JrbG9hZCInIEVSUgogICAgc2V0X3Byb3h5ICJodHRwIiAiaHR0cHMiCgogICAgY2F0IDw8RU9GID4gL29wdC9kbHZtL2NvbmZpZy5qc29uCiAgICB7CiAgICAgICJfY29tbWVudCI6ICJUaGlzIHByb3ZpZGVzIGRlZmF1bHQgc3VwcG9ydCBmb3IgUkFHOiBUZW5zb3JSVCBpbmZlcmVuY2UsIGxsYW1hMi0xM2IgbW9kZWwsIGFuZCBIMTAweDIgR1BVIiwKICAgICAgInJhZyI6IHsKICAgICAgICAib3JnX25hbWUiOiAiY29jZndnYThqcTJjIiwKICAgICAgICAib3JnX3RlYW1fbmFtZSI6ICJuby10ZWFtIiwKICAgICAgICAicmFnX3JlcG9fbmFtZSI6ICJudmlkaWEvcGFpZiIsCiAgICAgICAgImxsbV9yZXBvX25hbWUiOiAibnZpZGlhL25pbSIsCiAgICAgICAgImVtYmVkX3JlcG9fbmFtZSI6ICJudmlkaWEvbmVtby1yZXRyaWV2ZXIiLAogICAgICAgICJyYWdfbmFtZSI6ICJyYWctZG9ja2VyLWNvbXBvc2UiLAogICAgICAgICJyYWdfdmVyc2lvbiI6ICIyNC4wMyIsCiAgICAgICAgImVtYmVkX25hbWUiOiAibnYtZW1iZWQtcWEiLAogICAgICAgICJlbWJlZF90eXBlIjogIk5WLUVtYmVkLVFBIiwKICAgICAgICAiZW1iZWRfdmVyc2lvbiI6ICI0IiwKICAgICAgICAiaW5mZXJlbmNlX3R5cGUiOiAidHJ0IiwKICAgICAgICAibGxtX25hbWUiOiAibGxhbWEyLTEzYi1jaGF0IiwKICAgICAgICAibGxtX3ZlcnNpb24iOiAiaDEwMHgyX2ZwMTZfMjQuMDIiLAogICAgICAgICJudW1fZ3B1IjogIjIiLAogICAgICAgICJoZl90b2tlbiI6ICJodWdnaW5nZmFjZSB0b2tlbiB0byBwdWxsIGxsbSBtb2RlbCwgdXBkYXRlIHdoZW4gdXNpbmcgdmxsbSBpbmZlcmVuY2UiLAogICAgICAgICJoZl9yZXBvIjogImh1Z2dpbmdmYWNlIGxsbSBtb2RlbCByZXBvc2l0b3J5LCB1cGRhdGUgd2hlbiB1c2luZyB2bGxtIGluZmVyZW5jZSIKICAgICAgfQogICAgfQogICAgRU9GCiAgICBDT05GSUdfSlNPTj0kKGNhdCAiL29wdC9kbHZtL2NvbmZpZy5qc29uIikKICAgIElORkVSRU5DRV9UWVBFPSQoZWNobyAiJHtDT05GSUdfSlNPTn0iIHwganEgLXIgJy5yYWcuaW5mZXJlbmNlX3R5cGUnKQogICAgaWYgWyAiJHtJTkZFUkVOQ0VfVFlQRX0iID0gInRydCIgXTsgdGhlbgogICAgICByZXF1aXJlZF92YXJzPSgiT1JHX05BTUUiICJPUkdfVEVBTV9OQU1FIiAiUkFHX1JFUE9fTkFNRSIgIkxMTV9SRVBPX05BTUUiICJFTUJFRF9SRVBPX05BTUUiICJSQUdfTkFNRSIgIlJBR19WRVJTSU9OIiAiRU1CRURfTkFNRSIgIkVNQkVEX1RZUEUiICJFTUJFRF9WRVJTSU9OIiAiTExNX05BTUUiICJMTE1fVkVSU0lPTiIgIk5VTV9HUFUiKQogICAgZWxpZiBbICIke0lORkVSRU5DRV9UWVBFfSIgPSAidmxsbSIgXTsgdGhlbgogICAgICByZXF1aXJlZF92YXJzPSgiT1JHX05BTUUiICJPUkdfVEVBTV9OQU1FIiAiUkFHX1JFUE9fTkFNRSIgIkxMTV9SRVBPX05BTUUiICJFTUJFRF9SRVBPX05BTUUiICJSQUdfTkFNRSIgIlJBR19WRVJTSU9OIiAiRU1CRURfTkFNRSIgIkVNQkVEX1RZUEUiICJFTUJFRF9WRVJTSU9OIiAiTExNX05BTUUiICJOVU1fR1BVIiAiSEZfVE9LRU4iICJIRl9SRVBPIikKICAgIGVsc2UKICAgICAgZXJyb3JfZXhpdCAiSW5mZXJlbmNlIHR5cGUgJyR7SU5GRVJFTkNFX1RZUEV9JyBpcyBub3QgcmVjb2duaXplZC4gTm8gYWN0aW9uIHdpbGwgYmUgdGFrZW4uIgogICAgZmkKICAgIGZvciBpbmRleCBpbiAiJHshcmVxdWlyZWRfdmFyc1tAXX0iOyBkbwogICAgICBrZXk9IiR7cmVxdWlyZWRfdmFyc1skaW5kZXhdfSIKICAgICAganFfcXVlcnk9Ii5yYWcuJHtrZXksLH0gfCBzZWxlY3QgKC4hPW51bGwpIgogICAgICB2YWx1ZT0kKGVjaG8gIiR7Q09ORklHX0pTT059IiB8IGpxIC1yICIke2pxX3F1ZXJ5fSIpCiAgICAgIGlmIFtbIC16ICIke3ZhbHVlfSIgXV07IHRoZW4gCiAgICAgICAgZXJyb3JfZXhpdCAiJHtrZXl9IGlzIHJlcXVpcmVkIGJ1dCBub3Qgc2V0LiIKICAgICAgZWxzZQogICAgICAgIGV2YWwgJHtrZXl9PVwiIiR7dmFsdWV9IlwiCiAgICAgIGZpCiAgICBkb25lCgogICAgUkFHX1VSST0iJHtSQUdfUkVQT19OQU1FfS8ke1JBR19OQU1FfToke1JBR19WRVJTSU9OfSIKICAgIEVNQkVEX01PREVMX1VSST0iJHtFTUJFRF9SRVBPX05BTUV9LyR7RU1CRURfTkFNRX06JHtFTUJFRF9WRVJTSU9OfSIKCiAgICBOR0NfQ0xJX1ZFUlNJT049IjMuNDEuMiIKICAgIE5HQ19DTElfVVJMPSJodHRwczovL2FwaS5uZ2MubnZpZGlhLmNvbS92Mi9yZXNvdXJjZXMvbnZpZGlhL25nYy1hcHBzL25nY19jbGkvdmVyc2lvbnMvJHtOR0NfQ0xJX1ZFUlNJT059L2ZpbGVzL25nY2NsaV9saW51eC56aXAiCgogICAgbWtkaXIgLXAgL29wdC9kYXRhCiAgICBjZCAvb3B0L2RhdGEKCiAgICBpZiBbICEgLWYgLmZpbGVfZG93bmxvYWRlZCBdOyB0aGVuCiAgICAgICMgY2xlYW4gdXAKICAgICAgcm0gLXJmIGNvbXBvc2UuZW52ICR7UkFHX05BTUV9KiAke0xMTV9OQU1FfSogbmdjKiAke0VNQkVEX05BTUV9KiAqLmpzb24gLmZpbGVfZG93bmxvYWRlZAoKICAgICAgIyBpbnN0YWxsIG5nYy1jbGkKICAgICAgd2dldCAtLWNvbnRlbnQtZGlzcG9zaXRpb24gJHtOR0NfQ0xJX1VSTH0gLU8gbmdjY2xpX2xpbnV4LnppcCAmJiB1bnppcCBuZ2NjbGlfbGludXguemlwCiAgICAgIGV4cG9ydCBQQVRIPWBwd2RgL25nYy1jbGk6JHtQQVRIfQoKICAgICAgQVBJS0VZPSIiCiAgICAgIFJFR19VUkk9Im52Y3IuaW8iCgogICAgICBpZiBbWyAiJChncmVwIHJlZ2lzdHJ5LXVyaSAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpIiA9PSAqIiR7UkVHX1VSSX0iKiBdXTsgdGhlbgogICAgICAgIEFQSUtFWT0kKGdyZXAgcmVnaXN0cnktcGFzc3dkIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKICAgICAgZmkKCiAgICAgIGlmIFsgLXogIiR7QVBJS0VZfSIgXTsgdGhlbgogICAgICAgICAgZXJyb3JfZXhpdCAiTm8gQVBJS0VZIGZvdW5kIgogICAgICBmaQoKICAgICAgIyBjb25maWcgbmdjLWNsaQogICAgICBta2RpciAtcCB+Ly5uZ2MKCiAgICAgIGNhdCA8PCBFT0YgPiB+Ly5uZ2MvY29uZmlnCiAgICAgIFtDVVJSRU5UXQogICAgICBhcGlrZXkgPSAke0FQSUtFWX0KICAgICAgZm9ybWF0X3R5cGUgPSBhc2NpaQogICAgICBvcmcgPSAke09SR19OQU1FfQogICAgICB0ZWFtID0gJHtPUkdfVEVBTV9OQU1FfQogICAgICBhY2UgPSBuby1hY2UKICAgIEVPRgoKICAgICAgIyBuZ2MgZG9ja2VyIGxvZ2luCiAgICAgIGRvY2tlciBsb2dpbiBudmNyLmlvIC11IFwkb2F1dGh0b2tlbiAtcCAke0FQSUtFWX0KCiAgICAgICMgZG9ja2VyaHViIGxvZ2luIGZvciBnZW5lcmFsIGNvbXBvbmVudHMsIGUuZy4gbWluaW8KICAgICAgRE9DS0VSSFVCX1VSST0kKGdyZXAgcmVnaXN0cnktMi11cmkgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQogICAgICBET0NLRVJIVUJfVVNFUk5BTUU9JChncmVwIHJlZ2lzdHJ5LTItdXNlciAvb3B0L2Rsdm0vb3ZmLWVudi54bWwgfCBzZWQgLW4gJ3MvLipvZTp2YWx1ZT0iXChbXiJdKlwpLiovXDEvcCcpCiAgICAgIERPQ0tFUkhVQl9QQVNTV09SRD0kKGdyZXAgcmVnaXN0cnktMi1wYXNzd2QgL29wdC9kbHZtL292Zi1lbnYueG1sIHwgc2VkIC1uICdzLy4qb2U6dmFsdWU9IlwoW14iXSpcKS4qL1wxL3AnKQoKICAgICAgaWYgW1sgLW4gIiR7RE9DS0VSSFVCX1VTRVJOQU1FfSIgJiYgLW4gIiR7RE9DS0VSSFVCX1BBU1NXT1JEfSIgXV07IHRoZW4KICAgICAgICBkb2NrZXIgbG9naW4gLXUgJHtET0NLRVJIVUJfVVNFUk5BTUV9IC1wICR7RE9DS0VSSFVCX1BBU1NXT1JEfQogICAgICBlbHNlCiAgICAgICAgZWNobyAiV2FybmluZzogRG9ja2VySHViIG5vdCBsb2dpbiIKICAgICAgZmkKCiAgICAgICMgZ2V0IFJBRyBmaWxlcwogICAgICBuZ2MgcmVnaXN0cnkgcmVzb3VyY2UgZG93bmxvYWQtdmVyc2lvbiAke1JBR19VUkl9CgogICAgICAjIGdldCBsbG0gbW9kZWwKICAgICAgaWYgWyAiJHtJTkZFUkVOQ0VfVFlQRX0iID0gInRydCIgXTsgdGhlbgogICAgICAgIExMTV9NT0RFTF9VUkk9IiR7TExNX1JFUE9fTkFNRX0vJHtMTE1fTkFNRX06JHtMTE1fVkVSU0lPTn0iCiAgICAgICAgbmdjIHJlZ2lzdHJ5IG1vZGVsIGRvd25sb2FkLXZlcnNpb24gJHtMTE1fTU9ERUxfVVJJfQogICAgICAgIGNobW9kIC1SIG8rclggJHtMTE1fTkFNRX1fdiR7TExNX1ZFUlNJT059CiAgICAgICAgTExNX01PREVMX0ZPTERFUj0iL29wdC9kYXRhLyR7TExNX05BTUV9X3Yke0xMTV9WRVJTSU9OfSIKICAgICAgZWxpZiBbICIke0lORkVSRU5DRV9UWVBFfSIgPSAidmxsbSIgXTsgdGhlbgogICAgICAgIHBpcCBpbnN0YWxsIGh1Z2dpbmdmYWNlX2h1YgogICAgICAgIGh1Z2dpbmdmYWNlLWNsaSBsb2dpbiAtLXRva2VuICR7SEZfVE9LRU59CiAgICAgICAgaHVnZ2luZ2ZhY2UtY2xpIGRvd25sb2FkIC0tcmVzdW1lLWRvd25sb2FkICR7SEZfUkVQT30vJHtMTE1fTkFNRX0gLS1sb2NhbC1kaXIgJHtMTE1fTkFNRX0gLS1sb2NhbC1kaXItdXNlLXN5bWxpbmtzIEZhbHNlCiAgICAgICAgTExNX01PREVMX0ZPTERFUj0iL29wdC9kYXRhLyR7TExNX05BTUV9IgogICAgICAgIGNhdCA8PCBFT0YgPiAke0xMTV9NT0RFTF9GT0xERVJ9L21vZGVsX2NvbmZpZy55YW1sIAogICAgICAgIGVuZ2luZToKICAgICAgICAgIG1vZGVsOiAvbW9kZWwtc3RvcmUKICAgICAgICAgIGVuZm9yY2VfZWFnZXI6IGZhbHNlCiAgICAgICAgICBtYXhfY29udGV4dF9sZW5fdG9fY2FwdHVyZTogODE5MgogICAgICAgICAgbWF4X251bV9zZXFzOiAyNTYKICAgICAgICAgIGR0eXBlOiBmbG9hdDE2CiAgICAgICAgICB0ZW5zb3JfcGFyYWxsZWxfc2l6ZTogJHtOVU1fR1BVfQogICAgICAgICAgZ3B1X21lbW9yeV91dGlsaXphdGlvbjogMC44CiAgICBFT0YKICAgICAgICBjaG1vZCAtUiBvK3JYICR7TExNX01PREVMX0ZPTERFUn0KICAgICAgICBweXRob24zIC1jICJpbXBvcnQgeWFtbCwganNvbiwgc3lzOyBwcmludChqc29uLmR1bXBzKHlhbWwuc2FmZV9sb2FkKHN5cy5zdGRpbi5yZWFkKCkpKSkiIDwgIiR7UkFHX05BTUV9X3Yke1JBR19WRVJTSU9OfS9yYWctYXBwLXRleHQtY2hhdGJvdC55YW1sIj4gcmFnLWFwcC10ZXh0LWNoYXRib3QuanNvbgogICAgICAgIGpxICcuc2VydmljZXMuIm5lbW9sbG0taW5mZXJlbmNlIi5pbWFnZSA9ICJudmNyLmlvL252aWRpYS9uaW0vbmltX2xsbToyNC4wMi1kYXkwIiB8CiAgICAgICAgICAgIC5zZXJ2aWNlcy4ibmVtb2xsbS1pbmZlcmVuY2UiLmNvbW1hbmQgPSAibmltX3ZsbG0gLS1tb2RlbF9uYW1lICR7TU9ERUxfTkFNRX0gLS1tb2RlbF9jb25maWcgL21vZGVsLXN0b3JlL21vZGVsX2NvbmZpZy55YW1sIiB8CiAgICAgICAgICAgIC5zZXJ2aWNlcy4ibmVtb2xsbS1pbmZlcmVuY2UiLnBvcnRzICs9IFsiODAwMDo4MDAwIl0gfAogICAgICAgICAgICAuc2VydmljZXMuIm5lbW9sbG0taW5mZXJlbmNlIi5leHBvc2UgKz0gWyI4MDAwIl0nIHJhZy1hcHAtdGV4dC1jaGF0Ym90Lmpzb24gPiB0ZW1wLmpzb24gJiYgbXYgdGVtcC5qc29uIHJhZy1hcHAtdGV4dC1jaGF0Ym90Lmpzb24KICAgICAgICBweXRob24zIC1jICJpbXBvcnQgeWFtbCwganNvbiwgc3lzOyBwcmludCh5YW1sLnNhZmVfZHVtcChqc29uLmxvYWQoc3lzLnN0ZGluKSwgZGVmYXVsdF9mbG93X3N0eWxlPUZhbHNlLCBzb3J0X2tleXM9RmFsc2UpKSIgPCByYWctYXBwLXRleHQtY2hhdGJvdC5qc29uID4gIiR7UkFHX05BTUV9X3Yke1JBR19WRVJTSU9OfS9yYWctYXBwLXRleHQtY2hhdGJvdC55YW1sIgogICAgICBmaQoKICAgICAgIyBnZXQgZW1iZWRkaW5nIG1vZGVscwogICAgICBuZ2MgcmVnaXN0cnkgbW9kZWwgZG93bmxvYWQtdmVyc2lvbiAke0VNQkVEX01PREVMX1VSSX0KICAgICAgY2htb2QgLVIgbytyWCAke0VNQkVEX05BTUV9X3Yke0VNQkVEX1ZFUlNJT059CgogICAgICAjIGNvbmZpZyBjb21wb3NlLmVudgogICAgICBjYXQgPDwgRU9GID4gY29tcG9zZS5lbnYKICAgICAgZXhwb3J0IE1PREVMX0RJUkVDVE9SWT0iJHtMTE1fTU9ERUxfRk9MREVSfSIKICAgICAgZXhwb3J0IE1PREVMX05BTUU9JHtMTE1fTkFNRX0KICAgICAgZXhwb3J0IE5VTV9HUFU9JHtOVU1fR1BVfQogICAgICBleHBvcnQgQVBQX0NPTkZJR19GSUxFPS9kZXYvbnVsbAogICAgICBleHBvcnQgRU1CRURESU5HX01PREVMX0RJUkVDVE9SWT0iL29wdC9kYXRhLyR7RU1CRURfTkFNRX1fdiR7RU1CRURfVkVSU0lPTn0iCiAgICAgIGV4cG9ydCBFTUJFRERJTkdfTU9ERUxfTkFNRT0ke0VNQkVEX1RZUEV9CiAgICAgIGV4cG9ydCBFTUJFRERJTkdfTU9ERUxfQ0tQVF9OQU1FPSIke0VNQkVEX1RZUEV9LSR7RU1CRURfVkVSU0lPTn0ubmVtbyIKICAgIEVPRgoKICAgICAgdG91Y2ggLmZpbGVfZG93bmxvYWRlZAogICAgZmkKCiAgICAjIHN0YXJ0IE5HQyBSQUcKICAgIGRvY2tlciBjb21wb3NlIC1mICR7UkFHX05BTUV9X3Yke1JBR19WRVJTSU9OfS9kb2NrZXItY29tcG9zZS12ZWN0b3JkYi55YW1sIHVwIC1kIHBndmVjdG9yCiAgICBzb3VyY2UgY29tcG9zZS5lbnY7IGRvY2tlciBjb21wb3NlIC1mICR7UkFHX05BTUV9X3Yke1JBR19WRVJTSU9OfS9yYWctYXBwLXRleHQtY2hhdGJvdC55YW1sIHVwIC1kCgotIHBhdGg6IC9vcHQvZGx2bS91dGlscy5zaAogIHBlcm1pc3Npb25zOiAnMDc1NScKICBjb250ZW50OiB8CiAgICAjIS9iaW4vYmFzaAogICAgZXJyb3JfZXhpdCgpIHsKICAgICAgZWNobyAiRXJyb3I6ICQxIiA+JjIKICAgICAgdm10b29sc2QgLS1jbWQgImluZm8tc2V0IGd1ZXN0aW5mby52bXNlcnZpY2UuYm9vdHN0cmFwLmNvbmRpdGlvbiBmYWxzZSwgRExXb3JrbG9hZEZhaWx1cmUsICQxIgogICAgICBleGl0IDEKICAgIH0KCiAgICBjaGVja19wcm90b2NvbCgpIHsKICAgICAgbG9jYWwgcHJveHlfdXJsPSQxCiAgICAgIHNoaWZ0CiAgICAgIGxvY2FsIHN1cHBvcnRlZF9wcm90b2NvbHM9KCIkQCIpCiAgICAgIGlmIFtbIC1uICIke3Byb3h5X3VybH0iIF1dOyB0aGVuCiAgICAgICAgbG9jYWwgcHJvdG9jb2w9JChlY2hvICIke3Byb3h5X3VybH0iIHwgYXdrIC1GICc6Ly8nICd7aWYgKE5GID4gMSkgcHJpbnQgJDE7IGVsc2UgcHJpbnQgIiJ9JykKICAgICAgICBpZiBbIC16ICIkcHJvdG9jb2wiIF07IHRoZW4KICAgICAgICAgIGVjaG8gIk5vIHNwZWNpZmljIHByb3RvY29sIHByb3ZpZGVkLiBTa2lwcGluZyBwcm90b2NvbCBjaGVjay4iCiAgICAgICAgICByZXR1cm4gMAogICAgICAgIGZpCiAgICAgICAgbG9jYWwgcHJvdG9jb2xfaW5jbHVkZWQ9ZmFsc2UKICAgICAgICBmb3IgdmFyIGluICIke3N1cHBvcnRlZF9wcm90b2NvbHNbQF19IjsgZG8KICAgICAgICAgIGlmIFtbICIke3Byb3RvY29sfSIgPT0gIiR7dmFyfSIgXV07IHRoZW4KICAgICAgICAgICAgcHJvdG9jb2xfaW5jbHVkZWQ9dHJ1ZQogICAgICAgICAgICBicmVhawogICAgICAgICAgZmkKICAgICAgICBkb25lCiAgICAgICAgaWYgW1sgIiR7cHJvdG9jb2xfaW5jbHVkZWR9IiA9PSBmYWxzZSBdXTsgdGhlbgogICAgICAgICAgZXJyb3JfZXhpdCAiVW5zdXBwb3J0ZWQgcHJvdG9jb2w6ICR7cHJvdG9jb2x9LiBTdXBwb3J0ZWQgcHJvdG9jb2xzIGFyZTogJHtzdXBwb3J0ZWRfcHJvdG9jb2xzWypdfSIKICAgICAgICBmaQogICAgICBmaQogICAgfQoKICAgICMgJEA6IGxpc3Qgb2Ygc3VwcG9ydGVkIHByb3RvY29scwogICAgc2V0X3Byb3h5KCkgewogICAgICBsb2NhbCBzdXBwb3J0ZWRfcHJvdG9jb2xzPSgiJEAiKQoKICAgICAgQ09ORklHX0pTT05fQkFTRTY0PSQoZ3JlcCAnY29uZmlnLWpzb24nIC9vcHQvZGx2bS9vdmYtZW52LnhtbCB8IHNlZCAtbiAncy8uKm9lOnZhbHVlPSJcKFteIl0qXCkuKi9cMS9wJykKICAgICAgQ09ORklHX0pTT049JChlY2hvICR7Q09ORklHX0pTT05fQkFTRTY0fSB8IGJhc2U2NCAtLWRlY29kZSkKCiAgICAgIEhUVFBfUFJPWFlfVVJMPSQoZWNobyAiJHtDT05GSUdfSlNPTn0iIHwganEgLXIgJy5odHRwX3Byb3h5IC8vIGVtcHR5JykKICAgICAgSFRUUFNfUFJPWFlfVVJMPSQoZWNobyAiJHtDT05GSUdfSlNPTn0iIHwganEgLXIgJy5odHRwc19wcm94eSAvLyBlbXB0eScpCiAgICAgIGlmIFtbICQ/IC1uZSAwIHx8ICgteiAiJHtIVFRQX1BST1hZX1VSTH0iICYmIC16ICIke0hUVFBTX1BST1hZX1VSTH0iKSBdXTsgdGhlbgogICAgICAgIGVjaG8gIkluZm86IFRoZSBjb25maWctanNvbiB3YXMgcGFyc2VkLCBidXQgbm8gcHJveHkgc2V0dGluZ3Mgd2VyZSBmb3VuZC4iCiAgICAgICAgcmV0dXJuIDAKICAgICAgZmkKCiAgICAgIGNoZWNrX3Byb3RvY29sICIke0hUVFBfUFJPWFlfVVJMfSIgIiR7c3VwcG9ydGVkX3Byb3RvY29sc1tAXX0iCiAgICAgIGNoZWNrX3Byb3RvY29sICIke0hUVFBTX1BST1hZX1VSTH0iICIke3N1cHBvcnRlZF9wcm90b2NvbHNbQF19IgoKICAgICAgaWYgISBncmVwIC1xICdodHRwX3Byb3h5JyAvZXRjL2Vudmlyb25tZW50OyB0aGVuCiAgICAgICAgZWNobyAiZXhwb3J0IGh0dHBfcHJveHk9JHtIVFRQX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgaHR0cHNfcHJveHk9JHtIVFRQU19QUk9YWV9VUkx9CiAgICAgICAgZXhwb3J0IEhUVFBfUFJPWFk9JHtIVFRQX1BST1hZX1VSTH0KICAgICAgICBleHBvcnQgSFRUUFNfUFJPWFk9JHtIVFRQU19QUk9YWV9VUkx9CiAgICAgICAgZXhwb3J0IG5vX3Byb3h5PWxvY2FsaG9zdCwxMjcuMC4wLjEiID4+IC9ldGMvZW52aXJvbm1lbnQKICAgICAgICBzb3VyY2UgL2V0Yy9lbnZpcm9ubWVudAogICAgICBmaQogICAgICAKICAgICAgIyBDb25maWd1cmUgRG9ja2VyIHRvIHVzZSBhIHByb3h5CiAgICAgIG1rZGlyIC1wIC9ldGMvc3lzdGVtZC9zeXN0ZW0vZG9ja2VyLnNlcnZpY2UuZAogICAgICBlY2hvICJbU2VydmljZV0KICAgICAgRW52aXJvbm1lbnQ9XCJIVFRQX1BST1hZPSR7SFRUUF9QUk9YWV9VUkx9XCIKICAgICAgRW52aXJvbm1lbnQ9XCJIVFRQU19QUk9YWT0ke0hUVFBTX1BST1hZX1VSTH1cIgogICAgICBFbnZpcm9ubWVudD1cIk5PX1BST1hZPWxvY2FsaG9zdCwxMjcuMC4wLjFcIiIgPiAvZXRjL3N5c3RlbWQvc3lzdGVtL2RvY2tlci5zZXJ2aWNlLmQvcHJveHkuY29uZgogICAgICBzeXN0ZW1jdGwgZGFlbW9uLXJlbG9hZAogICAgICBzeXN0ZW1jdGwgcmVzdGFydCBkb2NrZXIKCiAgICAgIGVjaG8gIkluZm86IGRvY2tlciBhbmQgc3lzdGVtIGVudmlyb25tZW50IGFyZSBub3cgY29uZmlndXJlZCB0byB1c2UgdGhlIHByb3h5IHNldHRpbmdzIgogICAgfQ== che corrisponde allo script seguente in formato testo normale: #cloud-config write_files: - path: /opt/dlvm/dl_app.sh permissions: '0755' content: | #!/bin/bash set -eu source /opt/dlvm/utils.sh trap 'error_exit "Unexpected error occurs at dl workload"' ERR set_proxy "http" "https" cat <<EOF > /opt/dlvm/config.json { "_comment": "This provides default support for RAG: TensorRT inference, llama2-13b model, and H100x2 GPU", "rag": { "org_name": "cocfwga8jq2c", "org_team_name": "no-team", "rag_repo_name": "nvidia/paif", "llm_repo_name": "nvidia/nim", "embed_repo_name": "nvidia/nemo-retriever", "rag_name": "rag-docker-compose", "rag_version": "24.03", "embed_name": "nv-embed-qa", "embed_type": "NV-Embed-QA", "embed_version": "4", "inference_type": "trt", "llm_name": "llama2-13b-chat", "llm_version": "h100x2_fp16_24.02", "num_gpu": "2", "hf_token": "huggingface token to pull llm model, update when using vllm inference", "hf_repo": "huggingface llm model repository, update when using vllm inference" } } EOF CONFIG_JSON=$(cat "/opt/dlvm/config.json") INFERENCE_TYPE=$(echo "${CONFIG_JSON}" | jq -r '.rag.inference_type') if [ "${INFERENCE_TYPE}" = "trt" ]; then required_vars=("ORG_NAME" "ORG_TEAM_NAME" "RAG_REPO_NAME" "LLM_REPO_NAME" "EMBED_REPO_NAME" "RAG_NAME" "RAG_VERSION" "EMBED_NAME" "EMBED_TYPE" "EMBED_VERSION" "LLM_NAME" "LLM_VERSION" "NUM_GPU") elif [ "${INFERENCE_TYPE}" = "vllm" ]; then required_vars=("ORG_NAME" "ORG_TEAM_NAME" "RAG_REPO_NAME" "LLM_REPO_NAME" "EMBED_REPO_NAME" "RAG_NAME" "RAG_VERSION" "EMBED_NAME" "EMBED_TYPE" "EMBED_VERSION" "LLM_NAME" "NUM_GPU" "HF_TOKEN" "HF_REPO") else error_exit "Inference type '${INFERENCE_TYPE}' is not recognized. No action will be taken." fi for index in "${!required_vars[@]}"; do key="${required_vars[$index]}" jq_query=".rag.${key,,} | select (.!=null)" value=$(echo "${CONFIG_JSON}" | jq -r "${jq_query}") if [[ -z "${value}" ]]; then error_exit "${key} is required but not set." else eval ${key}=\""${value}"\" fi done RAG_URI="${RAG_REPO_NAME}/${RAG_NAME}:${RAG_VERSION}" EMBED_MODEL_URI="${EMBED_REPO_NAME}/${EMBED_NAME}:${EMBED_VERSION}" NGC_CLI_VERSION="3.41.2" NGC_CLI_URL="https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/${NGC_CLI_VERSION}/files/ngccli_linux.zip" mkdir -p /opt/data cd /opt/data if [ ! -f .file_downloaded ]; then # clean up rm -rf compose.env ${RAG_NAME}* ${LLM_NAME}* ngc* ${EMBED_NAME}* *.json .file_downloaded # install ngc-cli wget --content-disposition ${NGC_CLI_URL} -O ngccli_linux.zip && unzip ngccli_linux.zip export PATH=`pwd`/ngc-cli:${PATH} APIKEY="" REG_URI="nvcr.io" if [[ "$(grep registry-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p')" == *"${REG_URI}"* ]]; then APIKEY=$(grep registry-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') fi if [ -z "${APIKEY}" ]; then error_exit "No APIKEY found" fi # config ngc-cli mkdir -p ~/.ngc cat << EOF > ~/.ngc/config [CURRENT] apikey = ${APIKEY} format_type = ascii org = ${ORG_NAME} team = ${ORG_TEAM_NAME} ace = no-ace EOF # ngc docker login docker login nvcr.io -u \$oauthtoken -p ${APIKEY} # dockerhub login for general components, e.g. minio DOCKERHUB_URI=$(grep registry-2-uri /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') DOCKERHUB_USERNAME=$(grep registry-2-user /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') DOCKERHUB_PASSWORD=$(grep registry-2-passwd /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') if [[ -n "${DOCKERHUB_USERNAME}" && -n "${DOCKERHUB_PASSWORD}" ]]; then docker login -u ${DOCKERHUB_USERNAME} -p ${DOCKERHUB_PASSWORD} else echo "Warning: DockerHub not login" fi # get RAG files ngc registry resource download-version ${RAG_URI} # get llm model if [ "${INFERENCE_TYPE}" = "trt" ]; then LLM_MODEL_URI="${LLM_REPO_NAME}/${LLM_NAME}:${LLM_VERSION}" ngc registry model download-version ${LLM_MODEL_URI} chmod -R o+rX ${LLM_NAME}_v${LLM_VERSION} LLM_MODEL_FOLDER="/opt/data/${LLM_NAME}_v${LLM_VERSION}" elif [ "${INFERENCE_TYPE}" = "vllm" ]; then pip install huggingface_hub huggingface-cli login --token ${HF_TOKEN} huggingface-cli download --resume-download ${HF_REPO}/${LLM_NAME} --local-dir ${LLM_NAME} --local-dir-use-symlinks False LLM_MODEL_FOLDER="/opt/data/${LLM_NAME}" cat << EOF > ${LLM_MODEL_FOLDER}/model_config.yaml engine: model: /model-store enforce_eager: false max_context_len_to_capture: 8192 max_num_seqs: 256 dtype: float16 tensor_parallel_size: ${NUM_GPU} gpu_memory_utilization: 0.8 EOF chmod -R o+rX ${LLM_MODEL_FOLDER} python3 -c "import yaml, json, sys; print(json.dumps(yaml.safe_load(sys.stdin.read())))" < "${RAG_NAME}_v${RAG_VERSION}/rag-app-text-chatbot.yaml"> rag-app-text-chatbot.json jq '.services."nemollm-inference".image = "nvcr.io/nvidia/nim/nim_llm:24.02-day0" | .services."nemollm-inference".command = "nim_vllm --model_name ${MODEL_NAME} --model_config /model-store/model_config.yaml" | .services."nemollm-inference".ports += ["8000:8000"] | .services."nemollm-inference".expose += ["8000"]' rag-app-text-chatbot.json > temp.json && mv temp.json rag-app-text-chatbot.json python3 -c "import yaml, json, sys; print(yaml.safe_dump(json.load(sys.stdin), default_flow_style=False, sort_keys=False))" < rag-app-text-chatbot.json > "${RAG_NAME}_v${RAG_VERSION}/rag-app-text-chatbot.yaml" fi # get embedding models ngc registry model download-version ${EMBED_MODEL_URI} chmod -R o+rX ${EMBED_NAME}_v${EMBED_VERSION} # config compose.env cat << EOF > compose.env export MODEL_DIRECTORY="${LLM_MODEL_FOLDER}" export MODEL_NAME=${LLM_NAME} export NUM_GPU=${NUM_GPU} export APP_CONFIG_FILE=/dev/null export EMBEDDING_MODEL_DIRECTORY="/opt/data/${EMBED_NAME}_v${EMBED_VERSION}" export EMBEDDING_MODEL_NAME=${EMBED_TYPE} export EMBEDDING_MODEL_CKPT_NAME="${EMBED_TYPE}-${EMBED_VERSION}.nemo" EOF touch .file_downloaded fi # start NGC RAG docker compose -f ${RAG_NAME}_v${RAG_VERSION}/docker-compose-vectordb.yaml up -d pgvector source compose.env; docker compose -f ${RAG_NAME}_v${RAG_VERSION}/rag-app-text-chatbot.yaml up -d - path: /opt/dlvm/utils.sh permissions: '0755' content: | #!/bin/bash error_exit() { echo "Error: $1" >&2 vmtoolsd --cmd "info-set guestinfo.vmservice.bootstrap.condition false, DLWorkloadFailure, $1" exit 1 } check_protocol() { local proxy_url=$1 shift local supported_protocols=("$@") if [[ -n "${proxy_url}" ]]; then local protocol=$(echo "${proxy_url}" | awk -F '://' '{if (NF > 1) print $1; else print ""}') if [ -z "$protocol" ]; then echo "No specific protocol provided. Skipping protocol check." return 0 fi local protocol_included=false for var in "${supported_protocols[@]}"; do if [[ "${protocol}" == "${var}" ]]; then protocol_included=true break fi done if [[ "${protocol_included}" == false ]]; then error_exit "Unsupported protocol: ${protocol}. Supported protocols are: ${supported_protocols[*]}" fi fi } # $@: list of supported protocols set_proxy() { local supported_protocols=("$@") CONFIG_JSON_BASE64=$(grep 'config-json' /opt/dlvm/ovf-env.xml | sed -n 's/.*oe:value="\([^"]*\).*/\1/p') CONFIG_JSON=$(echo ${CONFIG_JSON_BASE64} | base64 --decode) HTTP_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.http_proxy // empty') HTTPS_PROXY_URL=$(echo "${CONFIG_JSON}" | jq -r '.https_proxy // empty') if [[ $? -ne 0 || (-z "${HTTP_PROXY_URL}" && -z "${HTTPS_PROXY_URL}") ]]; then echo "Info: The config-json was parsed, but no proxy settings were found." return 0 fi check_protocol "${HTTP_PROXY_URL}" "${supported_protocols[@]}" check_protocol "${HTTPS_PROXY_URL}" "${supported_protocols[@]}" if ! grep -q 'http_proxy' /etc/environment; then echo "export http_proxy=${HTTP_PROXY_URL} export https_proxy=${HTTPS_PROXY_URL} export HTTP_PROXY=${HTTP_PROXY_URL} export HTTPS_PROXY=${HTTPS_PROXY_URL} export no_proxy=localhost,127.0.0.1" >> /etc/environment source /etc/environment fi # Configure Docker to use a proxy mkdir -p /etc/systemd/system/docker.service.d echo "[Service] Environment=\"HTTP_PROXY=${HTTP_PROXY_URL}\" Environment=\"HTTPS_PROXY=${HTTPS_PROXY_URL}\" Environment=\"NO_PROXY=localhost,127.0.0.1\"" > /etc/systemd/system/docker.service.d/proxy.conf systemctl daemon-reload systemctl restart docker echo "Info: docker and system environment are now configured to use the proxy settings" } Immettere le proprietà di installazione del driver guest vGPU, ad esempio vgpu-license e nvidia-portal-api-key. Specificare i valori per le proprietà necessarie per un ambiente disconnesso in base alle esigenze. Vedere Proprietà OVF delle macchine virtuali di deep learning.
Output	Registri di installazione per il driver guest della vGPU in /var/log/vgpu-install.log. Per verificare che il driver guest della vGPU sia installato, accedere alla macchina virtuale tramite SSH ed eseguire il comando nvidia-smi. Registri dello script cloud-init in /var/log/dl.log. Per tenere traccia dello stato di avanzamento della distribuzione, eseguire tail -f /var/log/dl.log . Applicazione Web del chatbot di esempio a cui è possibile accedere all'indirizzo http://dl_vm_ip:3001/orgs/nvidia/models/text-qa-chatbot È possibile caricare la propria knowledge base.

Assegnazione di un indirizzo IP statico a una macchina virtuale di deep learning in VMware Private AI Foundation with NVIDIA

Per impostazione predefinita, le immagini della macchina virtuale di deep learning sono configurate con l'assegnazione dell'indirizzo DHCP. Se si desidera distribuire una macchina virtuale di deep learning con un indirizzo IP statico direttamente in un cluster vSphere, è necessario aggiungere ulteriore codice alla sezione cloud-init.

In vSphere with Tanzu, l'assegnazione dell'indirizzo IP è determinata dalla configurazione di rete per il supervisore in NSX.

Procedura

1. Creare uno script cloud-init in formato testo normale per il carico di lavoro DL che si intende utilizzare.
   Vedere Carichi di lavoro di deep learning in VMware Private AI Foundation with NVIDIA.
2. Inserire il codice aggiuntivo seguente nello script cloud-init.

   #cloud-config
   <instructions_for_your_DL_workload>
   
   manage_etc_hosts: true
    
   write_files:
     - path: /etc/netplan/50-cloud-init.yaml
       permissions: '0600'
       content: |
         network:
           version: 2
           renderer: networkd
           ethernets:
             ens33:
               dhcp4: false # disable DHCP4
               addresses: [x.x.x.x/x]  # Set the static IP address and mask
               routes:
                   - to: default
                     via: x.x.x.x # Configure gateway
               nameservers:
                 addresses: [x.x.x.x, x.x.x.x] # Provide the DNS server address. Separate mulitple DNS server addresses with commas.
    
   runcmd:
     - netplan apply

3. Codificare lo script cloud-init risultante in formato base64.
4. Impostare lo script cloud-init risultante in formato base64 come valore per il parametro OVF user-data dell'immagine della macchina virtuale di deep learning.

Esempio: Assegnazione di un indirizzo IP statico a un carico di lavoro di esempio CUDA

Per una macchina virtuale di deep learning di esempio con un carico di lavoro DL di esempio di CUDA:

Elemento macchina virtuale di deep learning	Valore di esempio
Immagine del carico di lavoro DL	nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8
Indirizzo IP	10.199.118.245
Prefisso subnet	/25
Gateway	10.199.118.253
Server DNS	10.142.7.1 10.132.7.1

si specifica il codice cloud-init seguente:

I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvb3B0L2Rsdm0vZGxfYXBwLnNoCiAgcGVybWlzc2lvbnM6ICcwNzU1JwogIGNvbnRlbnQ6IHwKICAgICMhL2Jpbi9iYXNoCiAgICBkb2NrZXIgcnVuIC1kIG52Y3IuaW8vbnZpZGlhL2s4cy9jdWRhLXNhbXBsZTp2ZWN0b3JhZGQtY3VkYTExLjcuMS11Ymk4CgptYW5hZ2VfZXRjX2hvc3RzOiB0cnVlCiAKd3JpdGVfZmlsZXM6CiAgLSBwYXRoOiAvZXRjL25ldHBsYW4vNTAtY2xvdWQtaW5pdC55YW1sCiAgICBwZXJtaXNzaW9uczogJzA2MDAnCiAgICBjb250ZW50OiB8CiAgICAgIG5ldHdvcms6CiAgICAgICAgdmVyc2lvbjogMgogICAgICAgIHJlbmRlcmVyOiBuZXR3b3JrZAogICAgICAgIGV0aGVybmV0czoKICAgICAgICAgIGVuczMzOgogICAgICAgICAgICBkaGNwNDogZmFsc2UgIyBkaXNhYmxlIERIQ1A0CiAgICAgICAgICAgIGFkZHJlc3NlczogWzEwLjE5OS4xMTguMjQ1LzI1XSAgIyBTZXQgdGhlIHN0YXRpYyBJUCBhZGRyZXNzIGFuZCBtYXNrCiAgICAgICAgICAgIHJvdXRlczoKICAgICAgICAgICAgICAgIC0gdG86IGRlZmF1bHQKICAgICAgICAgICAgICAgICAgdmlhOiAxMC4xOTkuMTE4LjI1MyAjIENvbmZpZ3VyZSBnYXRld2F5CiAgICAgICAgICAgIG5hbWVzZXJ2ZXJzOgogICAgICAgICAgICAgIGFkZHJlc3NlczogWzEwLjE0Mi43LjEsIDEwLjEzMi43LjFdICMgUHJvdmlkZSB0aGUgRE5TIHNlcnZlciBhZGRyZXNzLiBTZXBhcmF0ZSBtdWxpdHBsZSBETlMgc2VydmVyIGFkZHJlc3NlcyB3aXRoIGNvbW1hcy4KIApydW5jbWQ6CiAgLSBuZXRwbGFuIGFwcGx5

che corrisponde allo script seguente in formato testo normale:

#cloud-config
write_files:
- path: /opt/dlvm/dl_app.sh
  permissions: '0755'
  content: |
    #!/bin/bash
    docker run -d nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubi8

manage_etc_hosts: true
 
write_files:
  - path: /etc/netplan/50-cloud-init.yaml
    permissions: '0600'
    content: |
      network:
        version: 2
        renderer: networkd
        ethernets:
          ens33:
            dhcp4: false # disable DHCP4
            addresses: [10.199.118.245/25]  # Set the static IP address and mask
            routes:
                - to: default
                  via: 10.199.118.253 # Configure gateway
            nameservers:
              addresses: [10.142.7.1, 10.132.7.1] # Provide the DNS server address. Separate mulitple DNS server addresses with commas.
 
runcmd:
  - netplan apply

Configurazione di un'stanza di Deep Learning VM con un server proxy

Per connettere l'istanza di Deep Learning VM a Internet in un ambiente disconnesso in cui l'accesso a Internet viene eseguito tramite un server proxy, è necessario specificare i dettagli del server proxy nel file config.json nella macchina virtuale.

Procedura

1. Creare un file JSON con le proprietà per il server proxy.

   Server proxy che non richiede l'autenticazione	{ "http_proxy": "protocol://ip-address-or-fqdn:port", "https_proxy": "protocol://ip-address-or-fqdn:port" }
   Server proxy che richiede l'autenticazione	{ "http_proxy": "protocol://username:password@ip-address-or-fqdn:port", "https_proxy": "protocol://username:password@ip-address-or-fqdn:port" }

   dove:

   • protocol è il protocollo di comunicazione utilizzato dal server proxy, ad esempio http o https.
   • username e password sono le credenziali per l'autenticazione nel server proxy. Se il server proxy non richiede l'autenticazione, ignorare questi parametri.
   • ip-address-or-fqdn: indirizzo IP o nome host del server proxy.
   • port: numero della porta in cui il server proxy è in ascolto delle richieste in arrivo.
2. Codificare il codice JSON risultante in formato base64.
3. Quando si distribuisce l'immagine di Deep Learning VM, aggiungere il valore codificato alla proprietà OVF config-json.