Install Tanzu Kubernetes Grid Integrated Edition on VMware Cloud Foundation

This topic describes how to install and operate VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) on the VMware Cloud Foundation (VCF) platform.

Warning: Interoperability with VMware Cloud Foundation requires VCF v4.5.1 or later.

About TKGI Integration with VCF

VMware Cloud Foundation (VCF) is a unified SDDC platform that brings together vSphere, vSAN, NSX, and vRealize components into an integrated stack to deliver enterprise-ready infrastructure for private and public clouds. For more information, see the VCF Documentation.

You can install Tanzu Kubernetes Grid Integrated Edition (TKGI) on VCF. You can use either the TKGI Management Console or Ops Manager to install TKGI on VCF. The installation procedure on the VCF platform is generally the same as the installation procedure without VCF.

For more information, see:

• Requirements for Installing TKGI on VCF
• Supported Topologies for TKGI on VCF
• TKGI on VCF Deployment Procedure

Requirements for Installing TKGI on VCF

To install TKGI on VCF, you must adhere to the following requirements:

• Deploy only the supported versions of TKGI and VCF. See the Release Notes for precise version compatibility.

• Deploy vSphere 7.x with NSX-T 3.x using a converged VDS for vSphere and NSX-T traffic. You cannot use an N-VDS for NSX-T transport node traffic. Because of this requirement, a fresh installation is required. There is no way to migrate from N-VDS to VDS.

• Deploy TKGI in the Workload domain only. VCF creates domains, including a Management and Workload domain. TKGI components, including Ops Manager, BOSH Director, TKGI API and DB, and Harbor, must be installed into the Workload domain.

• Deploy TKGI using a single vSphere cluster, if you are using the TKGI Management Console. The reason is that currently the TKGI Management Console does not support using multiple converged VDSes. If you are using Ops Manager, you can use multiple vSphere clusters (with separate VDSes) as long as you are using a shared datastore (vSAN) to support PersistentVolumes.

• When deploying TKGI on VCF, do not generate a new certificate for NSX-T Manager. If you do it will break the VCF to NSX-T Manager communication. You must use the existing NSX-T Manager certificate when configuring TKGI.

• You must deploy the NSX-T Management plane in the VCF Management Domain using a VIP and an external load balancer in front of the NSX Manager nodes. See Configuring an NSX-T Management Plane Load Balancer for more information.

• You can use the Management Console or Ops Manager to deploy TKGI on VCF, but there is a dependency on the type of topology you want to deploy.

Supported Topologies for TKGI on VCF

TKGI supports two topologies for VCF: a Workload Domain with a single vSphere Cluster and a Workload Domain with multiple vSphere clusters.

Topology 1: Workload Domain with a Single vSphere Cluster

This topology provides a single vSphere cluster in the Workload Domain managed by the vCenter Server instance on the Management Domain. This is topology is supported by the TKGI Management Console and using Ops Manager to install TKGI.

Topology 2: Workload Domain with Multiple vSphere Clusters

This topology provides two or more vSphere clusters in the Workload Domain managed by the vCenter instance on the Management Domain. You must use Ops Manager to install TKGI for this topology. You cannot implement this topology using the TKGI Management Console.

TKGI on VCF Deployment Procedure

This section provides instructions for deploying TKGI once the Management and Workload Domains are created using VCF.

These instructions are high-level and assume hands-on experience deploying VCF, NSX-T, and TKGI. For assistance with installing VCF, see VMWare Cloud Foundation Deployment Guide. For assistance with installing NSX-T, see Installing and Configuring NSX-T Data Center v3.0 for Tanzu Kubernetes Grid Integrated Edition. For assistance with installing TKGI, see Installing Tanzu Kubernetes Grid Integrated Edition on vSphere.

Warning: Interoperability with VMware Cloud Foundation requires VCF v4.5.1 or later.

To deploy TKGI on VCF:

1. Install VCF v4.5.1 and deploy the Management Domain and a single Workload Domain. VCF installs and configures vSphere with vSAN and NSX-T. See the VMWare Cloud Foundation Deployment Guide for guidance.
2. Create one or two vSphere clusters in the Workload Domain depending on the chosen topology.
3. Log into the NSX-T Management plane in the Workload Domain and create the following NSX-T objects:
   • Configure a VLAN trunk port group as the network interface for each NSX-T Edge Node.
   • Configure an Uplink Profile for the Edge Nodes.
   • Deploy two Edge Nodes of size Large VM form factor.
   • Configure an Edge Cluster comprising the Edge Nodes.
4. If you are using Ops Manager to deploy TKGI, create the following additional NSX-T objects:
   • Configure a Tier-0 router and add the Edge Node uplink interfaces to get reachability to the physical switches.
   • Configure an Overlay logical switch for the deployment network.
   • Configure a Tier-1 router and add the interface to the Overlay logical switch. Redistribute connected routes.
   • Configure two IP blocks with mask /16: one for cluster nodes and one for cluster pods.
   • Configure one Floating IP pool for the Load Balancer VIP.
5. In vCenter, create two Resource Pool objects, one for deployment and one for Kubernetes node VMs. These will be used for Availability Zones.
6. Deploy the Ops Manager or TKGI Management Console OVA in the Management Domain on the management network.
7. Configure the tile or installer with the connection information for vCenter in the Workload Domain.
8. Configure the tile or installer with the connection information for the NSX-T Management plane in the Workload Domain.
9. Configure NSX-T networking as follows:
   • If you are using the Management Console, configure the Edge Cluster, Tier-0 router, IPAMs, and IP pools.
   • If you are using Ops Manager, provide IP addresses for Edge Node-1 uplink, Edge-Node-2 uplink, HA VIP for the uplink, and the default gateway.
   • Provide deployment network information.
   • Provide the Node and Pod CIDRs.
   • Provide the Load Balancer Floating IP range. NAT IP addresses will be taken from from this pool.
   • Get the Active cluster certificate from NSX-T and provide the same for the NSX certificate.
10. Create Availability Zones.
11. Select vSAN datastore for Ephemeral and Persistent storage.
12. Create Kubernetes plans.
13. Enable integration with vROPs and vRLI.
14. Enable Harbor. If you are using the Ops Manager installation, deploy Harbor after TKGI.
15. Deploy TKGI.
16. Test the TKGI on VCF deployment by installing the TKGI CLI, provisioning a Kubernetes cluster, and deploying a workload.