VMware Tanzu simplifies the operation of Kubernetes for multi-cloud deployment by centralizing management and governance for many clusters and teams across on-premises, public clouds, and edge. It delivers an open-source aligned Kubernetes distribution with consistent operations and management to support infrastructure and app modernization. This document provides a high-level architecture for deploying Tanzu editions. It also provides links for infrastructure provider specific reference architectures.
VMware Tanzu Kubernetes Grid - Enables creation and lifecycle management operations of Kubernetes clusters.
Contour Ingress Controller - Provides layer 7 ingress control to deployed HTTP(S) applications
NSX Advanced Load Balancer Essentials - Provides layer 4 service type Load Balancer support, recommended for vSphere deployments without NSX-T, or which have unique scale requirements.
Harbor Image Registry - Provides a centralized location to push, pull, store, and scan container images used in Kubernetes workloads. It also supports storing many other artifacts such as helm charts and includes enterprise grade features such as RBAC, retention policies, automated garbage clean up, and docker hub proxying among many other things.
VMware Tanzu Mission Control - Provides a global view of Kubernetes clusters, and allows for centralized policy management across all deployed and attached clusters.
Pinniped - Pinniped is an authentication service for Kubernetes clusters. This tool provides integration with identity providers such as OKTA, Dex and any LDAP identity provider.
VMware Tanzu Observability Extensions
Tanzu Kubernetes Grid allows you to create and manage ubiquitous Kubernetes clusters across multiple infrastructure providers using Kubernetes Cluster API. Tanzu Kubernetes Grid functions through the creation of a Management or Supervisor Kubernetes cluster which houses Cluster API. The Cluster API then interacts with the infrastructure provider to service workload Kubernetes cluster lifecycle requests.
Tanzu Editions includes components for observability, as well as container registry. It is generally recommended to install the necessary components into a centralized Shared Services cluster.
When targeting vSphere IaaS, additional convenient features are introduced in the integrated vSphere with Tanzu product; see VMware Tanzu Kubernetes Grid on vSphere Reference Design, which highlights and disambiguates these features.
The following design encompasses the generic network architecture for the Tanzu Kubernetes Grid reference design. For some infrastructure providers, you will find that the networks can be the same subnet or segments, and in other architectures might be entirely separate domains, but each infrastructure provider's networks can be mapped into this general framework.
Installing Tanzu Kubernetes Grid in an environment without internet access (air-gapped) requires staging the container images necessary for Tanzu into your environment. If allowed, an internet connected workstation with access to both networks can be used to stage the files from the Tanzu Network registry into a container registry such as Harbor.
In public infrastructure providers, internetless environments also require the creation of private endpoints for their respective services.
The reference architecture for each infrastructure provider covers the necessary steps to implement an air gapped installation into that specific environment. The Tanzu Kubernetes Grid documentation also provides instructions for copying container images into an environment.
docker-composeinstead of leveraging a Tanzu Kubernetes Grid shared-services cluster. This is ideal as the registry must be available prior to the bootstrap of management and subsequent workload clusters.
Pinniped consists of two components: a Supervisor and Concierge.
Examples of Identity Providers are:
See the following for more information on how to integrate an OIDC provider like OKTA and Pinniped into Tanzu Kubernetes Grid Tanzu Kubernetes Grid OIDC/LDAP Integration with Pinniped and OKTA OIDC Provider.
Velero is an open-source project from VMware enabling robust backup operations for Kubernetes clusters. Instead of infrastructure-centric backup strategies such as manual etcd snapshots, Velero targets Kubernetes objects and volumes via a set of controllers and custom resource definitions. This allows targeting Kubernetes-centric items such as namespaces, deployments and other cluster workloads, which can then be easily restored or even imported to different clusters.
In addition to Kubernetes objects, volumes can also be targeted for backup via cloud-provider specific plugins such as the
velero-vsphere plugin. This allows velero to perform volume snapshots, storing the contents in object storage for future restoration.
The following guide describes the installation and backup workflow of a cluster workload via the
For information about doing cluster backup with Velero, see Tanzu Kubernetes Grid Backup Operations with Velero vSphere Plugin.
Attaching clusters into global management through TMC allows you to manage your global portfolio of Kubernetes clusters. TMC can assist you with:
If the workload cluster put under management requires a proxy to access the internet, you can use the TMC CLI to generate the YAML necessary to install TMC components on it.
For a complete list of features that TMC includes with Tanzu see this chart.
Observability can be significantly enhanced by using Tanzu Observability by Wavefront. Wavefront is a VMware SaaS system which is used to collect and display metrics and trace data from the full stack platform as well as from applications. The service provides the ability to create alerts tuned by advanced analytics, assist in the troubleshooting of systems and to understand the impact of running production code.
In the case of vSphere and Tanzu Kubernetes Grid, Wavefront is used to collect data from components in vSphere, from Kubernetes, and from applications running within Kubernetes.
You can configure Wavefront with an array of capabilities. Here are the recommended plugins for this design:
|Plugin||Purpose||Key Metrics||Example Metrics|
|Wavefront Kubernetes Integration||Collect metrics from Kubernetes clusters and pods||Kubernetes container and POD statistics||POD CPU usage rate|
|Wavefront by VMware for Istio||Adapts Istio collected metrics and forwards to Wavefront||Istio metrics including request rates, trace rates, throughput, etc.||Request rate (Transactions per Second)|
Prometheus operates by exposing scrapable metrics endpoints for various monitoring targets throughout your cluster. Metrics are ingested by polling the endpoints on a set interval which are then stored in a time-series database. Metrics data can be explored via the Prometheus Query Language interface.
Grafana is responsible for vizualizing Prometheus metrics without the need to manually write
PromQL queries. Custom charts and graphs can be created in addition to the pre-packaged optons.
The Tanzu Kubernetes Grid extensions bundles contain instructions & manifests for deploying these tools out.
Tanzu also includes fluent bit for integration with logging platforms such as vRealize LogInsight and other logging aggregators. Details on configuring fluent bit to your logging provider can be found in the documentation here.
Follow the reference architecture for Tanzu Kubernetes Grid on vSphere here.