Load Balancers in Tanzu Kubernetes Grid Integrated Edition

This topic describes the VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) load balancers for the TKGI API and TKGI clusters and workloads.

Overview

Load balancers used with TKGI differ by the type of deployment:

Load Balancers in Tanzu Kubernetes Grid Integrated Edition Deployments without NSX
Load Balancers in Tanzu Kubernetes Grid Integrated Edition Deployments on vSphere with NSX

Load Balancers in Tanzu Kubernetes Grid Integrated Edition Deployments without NSX

For Tanzu Kubernetes Grid Integrated Edition deployments on GCP, AWS, or vSphere without NSX, you can configure load balancers for the following:

TKGI API Load Balancer: Configuring this load balancer enables you to run TKGI Command Line Interface (TKGI CLI) commands from your local workstation.
Kubernetes Cluster Load Balancers: Configuring a load balancer for each new cluster enables you to run Kubernetes CLI (kubectl) commands on the cluster.
Workload Load Balancers): Configuring a load balancer for your application workloads enables external access to the services that run on your cluster.

The following diagram, applicable to GCP, AWS, and vSphere without NSX, shows where each of the above load balancers can be used within your Tanzu Kubernetes Grid Integrated Edition deployment. TKGI load balancer diagram including all load balancer options for TKGI deployments without NSX

If you use either vSphere without NSX or GCP, you are expected to create your own load balancers within your cloud provider console. If your cloud provider does not offer load balancing, you can use any external TCP or HTTPS load balancer of your choice.

TKGI API Load Balancer

The TKGI API load balancer enables you to access the TKGI API from outside the network on Tanzu Kubernetes Grid Integrated Edition deployments on GCP, AWS, and on vSphere without NSX. For example, configuring a load balancer for the TKGI API enables you to run TKGI CLI commands from your local workstation.

For information about configuring the TKGI API load balancer on vSphere without NSX, see Configuring TKGI API Load Balancer.

Kubernetes Cluster Load Balancers

When you create an Tanzu Kubernetes Grid Integrated Edition cluster on GCP, AWS, and on vSphere without NSX, you must configure external access to the cluster by creating an external TCP or HTTPS load balancer. The load balancer enables the Kubernetes CLI to communicate with the cluster.

If you create a cluster in a non-production environment, you can choose not to use a load balancer. To enable kubectl to access the cluster without a load balancer, you can do one of the following:

Create a DNS entry that points to the cluster’s control plane VM. For example: my-cluster.example.com A 10.0.0.5
On the workstation where you run kubectl commands, add the control plane IP address of your cluster and kubo.internal to the /etc/hosts file. For example: 10.0.0.5 kubo.internal

For more information about configuring a cluster load balancer, see the following:

Workload Load Balancers

To enable external access to your Tanzu Kubernetes Grid Integrated Edition app on GCP, AWS, and on vSphere without NSX, you can either create a load balancer or expose a static port on your workload.

For information about configuring a load balancer for your app workload, see Deploying and Exposing Basic Linux Workloads.

If you use AWS, you must configure routing in the AWS console before you can create a load balancer for your workload. You must create a public subnet in each availability zone (AZ) where you are deploying the workload and tag the public subnet with your cluster’s unique identifier.

See the AWS Prerequisites section of Deploying and Exposing Basic Linux Workloads before you create a workload load balancer.

Deploy Your Workload Load Balancer with an Ingress Controller

A Kubernetes ingress controller sits behind a load balancer, routing HTTP and HTTPS requests from outside the cluster to services within the cluster. Kubernetes ingress resources can be configured to load balance traffic, provide externally reachable URLs to services, and manage other aspects of network traffic.

If you add an ingress controller to your Tanzu Kubernetes Grid Integrated Edition deployment, traffic routing is controlled by the ingress resource rules you define. VMware recommends configuring Tanzu Kubernetes Grid Integrated Edition deployments with both a workload load balancer and an ingress controller.

The following diagram shows how the ingress routing can be used within your Tanzu Kubernetes Grid Integrated Edition deployment. TKGI diagram that shows ingress routing for both Istio and NSX

The load balancer on Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX is automatically provisioned with Kubernetes ingress resources without the need to deploy and configure an additional ingress controller.

For information about deploying a load balancer configured with ingress routing on GCP, AWS, Azure, and vSphere without NSX, see Configuring Ingress Routing. For information about ingress routing on vSphere with NSX, see Configuring Ingress Resources and Load Balancer Services.

Load Balancers in Tanzu Kubernetes Grid Integrated Edition Deployments on vSphere with NSX

Tanzu Kubernetes Grid Integrated Edition deployments on vSphere with NSX in high-availability mode require you configure a load balancer to access the TKGI API. To configure an NSX load balancer for TKGI API traffic, see Provisioning an NSX Load Balancer for the TKGI API Server.

TKGI deployments on vSphere with NSX in singleton mode require you configure only a DNAT rule so that the TKGI API host is accessible. These TKGI deployments do not require you to configure a load balancer to access the TKGI API. For more information, see Share the Tanzu Kubernetes Grid Integrated Edition Endpoint in Installing Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX Integration.

At runtime, NSX automatically handles load balancer creation, configuration, and deletion as part of the Kubernetes cluster create, update, and delete process. When a new Kubernetes cluster is created, NSX creates and configures a dedicated load balancer tied to it. The load balancer is a shared resource designed to provide efficient traffic distribution to control plane nodes as well as services deployed on worker nodes. Each application service is mapped to a virtual server instance, carved out from the same load balancer. For more information, see Logical Load Balancer in the NSX-T documentation.

Virtual server instances are created on the load balancer to provide access to the following:

Kubernetes API and UI services on a Kubernetes cluster. This enables requests to be load balanced across multiple control plane nodes.
Ingress controller. This enables the virtual server instance to dispatch HTTP and HTTPS requests to services associated with Ingress rules.
type:loadbalancer services. This enables the server to handle TCP connections or UDP flows toward exposed services.

Load balancers are deployed in high-availability mode so that they are resilient to potential failures and able to recover quickly from critical conditions.

Note: The NodePort Service type is not supported for Tanzu Kubernetes Grid Integrated Edition deployments on vSphere with NSX. Only type:LoadBalancerServices and Services associated with Ingress rules are supported on vSphere with NSX.

Resizing Load Balancers

When a new Kubernetes cluster is provisioned using the TKGI API, NSX creates a dedicated load balancer for that new cluster. By default, the size of the load balancer is set to Small.

With network profiles, you can change the size of the load balancer deployed by NSX at the time of cluster creation. For information about network profiles, see Using Network Profiles (NSX Only).

For more information about the types of load balancers NSX provisions and their capacities, see Scaling Load Balancer Resources in the NSX-T documentation.