VMware Tanzu simplifies the operations of Kubernetes for multi-cloud deployments by centralizing management and governance for clusters and teams across on-premises, public clouds, and the edge. It delivers an open source aligned Kubernetes distribution with consistent operations and management to support infrastructure and app modernization.
This document provides a step-by-step guide for how to install and deploy Tanzu for Kubernetes operations on Microsoft Azure.
The scope of this document is limited to providing the deployment steps based on the following reference design. The reference design represents one of the two production-level reference designs described in VMware Tanzu for Kubernetes Operations on Azure Reference Design.
This design shows both the Tanzu Kubernetes Grid management cluster and workload clusters in the same virtual network along with the bootstrap machine. However, each cluster is placed in its own subnet. In addition, the control plane and worker nodes of each cluster are also separated by a subnet.
The reference design shows the deployment of only the base components within Tanzu Kubernetes Grid.
The reference design fits in with any production-level design that a customer may have in place, such as a Hub and Spoke, Global WAN Peering, or just a simple DMZ based implementation.
This guide does not make any assumptions about your chosen tooling for security or DevOps, other than what is available with a default Azure subscription.
Note: You can use this guide to deploy additional workload clusters and workload clusters of a different size. However, you’ll need to make additional configuration changes. You can make these configuration changes after you have gone through the deployment steps provided in this document.
Ensure that you have:
Before deploying Tanzu for Kubernetes operations and the actual Kubernetes clusters, ensure that your Azure environment is set up as described in this section.
The deployment detailed in this document uses the following resources:
The ARM template contains parameters that you can populate or customize so that your Azure environment uses your naming standards and networking requirements.
The ARM template deploys the following items:
In addition, the ARM template,
To successfully deploy Tanzu Kubernetes Grid to Azure, ensure that the quotas are sufficient to support both the management cluster and workload cluster deployments. Otherwise, the deployments will fail.
Review the quotas for the following resources, which are included in the ARM template, and increase their values as needed. Increase the quotas for every region to which you plan to deploy Tanzu Kubernetes Grid.
Based on the recommended minimum virtual machine size of D2s_v3, the following minimum quotas are required per cluster:
Ensure that you increase the quotas if you make changes to the basic configuration of the clusters.
There are multiple methods to deploy an ARM template on Azure. If you are experienced with Azure, you can deploy the ARM template in a method that is comfortable to you.
Otherwise, you can use the example Azure CLI commands locally or in Azure Cloud Shell. If you prefer to use Azure PowerShell, use the example command for Azure PowerShell.
Ensure that you have the following to deploy the ARM template in Microsoft Azure:
Run the following example Azure CLI command locally or in Azure Cloud Shell to deploy the ARM template.
az deployment create –template-file azure-deploy.json –parameters azure-deploy-parameters.json –resource-group <Resource Group Name>
Alternatively, run the following example command in Azure PowerShell to deploy the ARM template.
New-AzResourceGroupDeployment -ResourceGroupName <Resource Group Name> -TemplateFile azure-deploy.json -TemplateParameterFile azure-deploy-parameters.json
If you prefer to use the Azure Portal, do the following to process an ARM template directly on the Azure Portal.
Search and click Deploy a Custom Template > Build your own template in the editor.
Click Load file to upload the ARM template,
Fill in the parameter values so that the values are specific to your deployment.
The Tanzu CLI requires access to an Azure Service Principal (SP) or Application Registration to programmatically configure the Tanzu cluster’s infrastructure during deployment and during auto-scale events.
VMware recommends that you create the SP on the Azure Portal. However, if you prefer to use either the Azure CLI or Azure PowerShell, see the following Microsoft product documentation:
Important: To create an Azure Service Principal or Application Registration, you must be an Administrator in your Azure Active Directory tenant. Alternatively, all Users must have App Registrations set to Yes, which allows all Users to create an Azure Service Principal.
Do the following on the Azure portal to create an Azure Service Principal:
Go to Azure Active Directory > Application Registrations.
Click New App Registration.
Enter the following:
After the Application Registration is created, an Overview page appears.
Copy the values for the Application client ID and Directory (tenant) ID from the Overview page. You will need the IDs for running the Tanzu CLI.
Add a Key to the Application Registration.
You will use the key for programmatic authentication and execution.
Click Certificates & secrets > Client secrets > New client secret.
Choose the expiration date.
Store the randomly generated key so that it can be used later.
Assign VM Contributor and Network Contributor roles to the Azure SP.
The roles provide the minimum level of permissions required for the Tanzu CLI to function properly within Azure.
Assign the roles through the Subscription scope. Depending on your security boundaries, you can also assign it at the Resource Group scope.
Important: To assign a role to the SP, you must have either the Owner role or User Access Administration role within the scope of the Azure subscription.
Find your specific Subscription on the Azure Portal and go to Access Control (IAM) > Roles.
Click Add role assignment.
In the Add role assignment page, select User, group, or service principal.
For Select Members, search for the new SP name you created.
Make a note of the following information. You will need the information to create the configuration files to set up the Bootstrap machine and Tanzu CLI.
You will use the bootstrap VM to deploy the Tanzu Kubernetes Grid management and workload clusters. Create the bootstrap VM after you have set up your Microsoft Azure environment.
You will set up the bootstrap VM with the following:
To set up the bootstrap VM:
# Variables export VMWUSER = "<CUSTOMER_CONNECT_USERID>" export VMWPASS = "<CUSTOMER_CONNECT_PWD>" export AZURETENANTID = "<AAD Tenant ID>" export AZURESUBSCRIPTION = "<Subscription GUID>" export AZURECLIENTID = "<Service Principal ID>" export AZURECLIENTSECRET = "<Service Principal Secret>" sudo apt-get update sudo apt-get upgrade # Docker Install & Verify sudo apt-get install \ apt-transport-https \ ca-certificates \ curl \ gnupg \ lsb-release curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io sudo groupadd docker sudo usermod -aG docker $USER # Optional Verification # docker run hello-world # Downloading and Installing Tanzu CLI git clone https://github.com/z4ce/vmw-cli cd vmw-cli curl -o tmc 'https://tmc-cli.s3-us-west-2.amazonaws.com/tmc/0.4.3-fcb03104/linux/x64/tmc' ./vmw-cli ls ./vmw-cli ls vmware_tanzu_kubernetes_grid ./vmw-cli cp tanzu-cli-bundle-linux-amd64.tar.gz ./vmw-cli cp "$(./vmw-cli ls vmware_tanzu_kubernetes_grid | grep kubectl-linux | cut -d ' ' -f1)" curl -o yq -L https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 fi sudo install yq /usr/local/bin sudo install tmc /usr/local/bin/tmc yes | tar --overwrite -xzvf tanzu-cli-bundle-linux-amd64.tar.gz yes | gunzip kubectl-*.gz sudo install kubectl-linux-* /usr/local/bin/kubectl cd cli/ sudo install core/*/tanzu-core-linux_amd64 /usr/local/bin/tanzu yes | gunzip *.gz sudo install imgpkg-linux-amd64-* /usr/local/bin/imgpkg sudo install kapp-linux-amd64-* /usr/local/bin/kapp sudo install kbld-linux-amd64-* /usr/local/bin/kbld sudo install vendir-linux-amd64-* /usr/local/bin/vendir sudo install ytt-linux-amd64-* /usr/local/bin/ytt cd .. tanzu plugin sync tanzu config init # Azure CLI Install and VM Acceptance curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash az login --service-principal --username $AZURECLIENTID --password $AZURECLIENTSECRET --tenant $AZURETENANTID az vm image terms accept --publisher vmware-inc --offer tkg-capi --plan k8s-1dot21dot2-ubuntu-2004 --subscription $AZURESUBSCRIPTION
Note: Because of permission issues, you will have to log out and log in to the bootstrap VM after installing Docker and before you download and install the Tanzu components.
If you prefer not to copy paste code, you can use the following sample script files:
Deploy Tanzu Kubernetes Grid after you set up your Azure environment and bootstrap VM. You will use Tanzu CLI to deploy a management cluster and workload cluster.
Create a YAML file that contains the required configuration details.
The ex-config.yaml sample YAML file contains the minimum configuration needed to deploy a management cluster and workload clusters. The configuration contains the default values used in the ARM template. Change the values in the YAML as needed for your deployment. For example, replace the values for the Azure IDs, Application Registration/Service Principal, cluster name, and the Base 64 encoded value of the public key.
Run the following commands from your bootstrap VM to create the management and workload clusters.
tanzu management-cluster create --file config.yaml -v 0-9 tanzu cluster create –file config.yaml -v 0-9
For additional product documentation on how to create the YAML configuration file and what each value corresponds to in Azure, see Management Cluster Configuration for Microsoft Azure.
The following VMware SaaS services provide additional Kubernetes lifecycle management, observability, and service mesh features.
For configuration information, see Configure SaaS Services.
A package in Tanzu Kubernetes Grid is a collection of related software that supports or extends the core functionality of the Kubernetes cluster in which the package is installed.
These packages are available for deployment in each workload cluster that you deploy, but they are not automatically installed and working as pods.
Tanzu Kubernetes Grid includes two types of packages, auto-managed packages and CLI-managed packages.
A CLI-managed package is an optional component of a Kubernetes cluster that you can install and manage with the Tanzu CLI. These packages are installed after cluster creation. CLI-managed packages are grouped into package repositories in the Tanzu CLI. If a package repository that contains CLI-managed packages is available in the target cluster, you can use the Tanzu CLI to install and manage any of the packages from that repository.
Using the Tanzu CLI, you can install CLI-managed packages from the built-in
tanzu-standard package repository or from package repositories that you add to your target cluster. From the
tanzu-standard package repository, you can install the Cert Manager, Contour, External DNS, Fluent Bit, Grafana, Harbor, Multus CNI, and Prometheus packages. For more information about CLI-managed packages, see CLI-Managed Packages.
The following provide more information on installing VMware recommended CLI-managed packages:
If your deployment requires Harbor to take on a heavy load and store large images in the registry, you can install Harbor into a separate workload cluster.