Tanzu Application Engine Conceptual Overview

Tanzu Application Engine enables a separation of concerns and responsibilities between the individuals responsible for building an application platform (often platform engineers) and those deploying and using the platform (often application developers). To do this, Tanzu Application Engine relies on several key concepts that, when combined, provide the power of Tanzu Application Engine.

Spaces – Repeatable Application Runtimes

Spaces act as the focal point where contributions from all of the individuals responsible for building an application platform combine to enable users to run their applications efficiently and effectively. From the perspective of a Space user, Spaces appear to be the equivalent of a Kubernetes namespace: a container where applications and other resources are deployed that ensures boundaries such as permissions and networking discoverability. From the perspective of the application platform builders, Spaces take the namespace concept and make it much more powerful. Spaces become a way to control which APIs are available to users, where applications are deployed and their level of availability, and a way to guarantee consistent usage of platform-provided Capabilities.

The concepts that follow are how Spaces achieve this curated platform power.

Capabilities – Platform APIs and Features

Capabilities describe the APIs and features available from the platform, such as service-binding, ingress, fluxcd-helm. Depending on where they are referenced, Capabilities are used to achieve different things.

Spaces conceptual diagram showing Spaces, clusters, and Capabilities

To curate the APIs that are available to users, platform engineers install packages on clusters. These packages provide APIs or custom resource definitions (CRDs) and their respective controllers on the cluster, which enables functionality on the cluster. When a package that includes the capability.tanzu.vmware.com/provides annotation is installed on a cluster, that cluster is considered to provide a given Capability. Platform engineers are required to explicitly choose which Capabilities to provide, based on which Capability-providing packages they install. This enables them to control the types workloads that can be installed on different clusters.

When an application operator creates a Space, the applications that are deployed to that Space can use the Capabilities that are provided by the platform. However, Spaces by default do not have access to any APIs, CRDs, or Capabilities on the platform. To make the Capabilities available to a Space, a Space author must require Capabilities. A Space with no required Capabilities means applications in it are not able to use any Capabilities.

A Space will be scheduled on a cluster only if that cluster provides all of the Capabilities that the Space requires.

As an example, Spring Cloud Gateway is a Capability. A platform engineer at a company that uses Spring Cloud Gateway would install the Spring Cloud Gateway package on their clusters. The installation of this package installs various CRDs and controllers that can be used by other workloads on the cluster. The clusters have the ability to create Spring Cloud Gateways, via the CRDs from the Package, but no gateways are created yet.

Taking the example a bit further, if the platform engineer only installs the Spring Cloud Gateway on clusters designed for web application workloads and not their more expensive GPU clusters, they can ensure that web applications – which require Spring Cloud Gateway – will never land on their GPU clusters.

When a Space that requires the Spring Cloud Gateway Capability is created, the Space will be scheduled only on a Cluster that provides the Spring Cloud Gateway Capability. Additionally, the Capability’s CRDs are exposed to workloads inside of the Space so that application developers that are pushing code to the Space can use the various CRDs for building gateways in the Space.

Cluster Groups – Consistent and Reusable Provided Capabilities

Cluster groups are groups of provided Capabilities and the clusters that they are installed on. With cluster groups, platform builders can manage all clusters – regardless of Kubernetes distribution – in a given cluster group in a consistent manner. Platform builders use cluster groups to manage their clusters at a fleet level, rather than an individual basis; they are used to assign policies and install Capabilities.

Spaces conceptual diagram showing cluster groups added to prior diagram

Cluster groups install packages to provide Capabilities on clusters. More specifically, packages that have the capability.tanzu.vmware.com/provides annotation are Capability-providing packages. By using cluster groups to install Capability-providing packages, all clusters in a given cluster group provide the same Capabilities.

For example, a platform builder that is knowledgeable on how to deploy web applications might provide the “spring-cloud-gateway” and “service-binding” Capabilities in a “web application” cluster group. They are then able to add clusters across multiple cloud providers and regions, with clusters varying in both size and configuration. They do this knowing that all of these clusters will then have the same consistent set of packages installed on them and that they will all provide the same Capabilities for Spaces.

Cluster groups are intended to include many clusters. In the example above, the team building the platform is attempting to standardize the way that their organization configures clusters to support web applications. Therefore a TKG cluster and an EKS cluster should be in the same cluster group to ensure that they provide the same set of Capabilities. This consistency enables teams to update the packages installed on their clusters (for example, to respond to a security vulnerability) and be confident that the updates are rolled out to all clusters in the cluster group.

Availability Targets – Fault Domains and Locality

Availability Targets are groupings of clusters that allow developers to define where their application will be scheduled. They can generally be thought of as named, physical locations but they are not limited to that.

Spaces conceptual diagram showing adding Availability Target to prior diagram

Spaces specify Availability Targets to describe where they should be scheduled. By including Availability Targets, rather than clusters, platform engineers can group clusters into into logical groups that together provide Kubernetes capacity to run workloads. As a result, a space creator can schedule a space to a specific datacenter, site, or anything beyond, without ever needing to know which specific cluster it is running on.

For example, a common pattern is for customers to have several data centers in different cities, like Las Vegas and Charlotte, to provide redundancy in case of a disaster. A platform builder who wants to easily enable application operators to run applications in their datacenters might collect all clusters in their Las Vegas datacenter into a “primary" Availability Target and all clusters in their Charlotte data center into a “secondary” Availability Target. An application team, that is unaware of the details of how their cloud footprint is structured and what they would need to do to be reliable, can include the “primary” and “secondary” Availability Targets for their Space and be assured that the platform is running their application on clusters in both datacenters for them.

It is possible that no cluster exists in an Availability Target chosen by a Space. This is by design as it provides platform builders a way to prevent workloads that the platform may not be prepared for, from running in these locations. To remedy this situation, the platform builder creates a new cluster for the appropriate Availability Target, adding it to the pool to be scheduled on.

Space Scheduling

Tanzu Application Engine “schedules” Spaces onto clusters similar to the way that Kubernetes schedules pods onto nodes within a cluster. As the platform is constantly watching for changes to clusters being created and deleted, which Capabilities are provided by those clusters, and what each Space’s configuration looks like, Application Engine is constantly recalculating and updating, if necessary, where a Space is scheduled.

Conceptual diagram showing Space scheduling process

The required Capability and Availability Target metadata defined by each Space are passed through a multi-tier filtering algorithm to determine which clusters will host the Space’s contents that looks roughly like this:

  1. For each requested Availability Target, select all clusters.
  2. Filter that list of clusters to only ones that provide all Capabilities required by the Space.
  3. Filter that list of clusters to only ones that have sufficient resource capacity (for example, CPU and RAM) to host the Space.
  4. From that list, select a cluster and assign it to the Space for use.
    1. Note that the selection process is a stable algorithm; when a cluster is selected, the same cluster will be selected unless it fails to meet one of the criteria above.
  5. Create a namespace on the assigned cluster and begin continuously syncing Space content.

Continuing the examples from above, when a Space creator creates a Space that includes the Availability Targets “primary” and “secondary”, and requires the Certificate Manager Capability, the Tanzu Application Engine does the following:

  1. For “primary” Availability Target, find all of its clusters, which are Cluster A and Cluster B.
  2. Filter clusters that provide all of the Space’s required Capabilities, which is only Cluster A.
  3. Filter clusters with sufficient resource capacity for the Space; because the Space did not set resource limits, Cluster A has enough capacity.
  4. Select a matching cluster from the above steps. In this case, Cluster A is the only matching cluster, but if there were more, Tanzu would select one from the list.
  5. Create a namespace on Cluster A and sync content from Tanzu to the cluster.
  6. Repeat the above steps for the “secondary” Availability Target.

Profiles – Consistent and Reusable Required Capabilities

Profiles are groups of required Capabilities; they are intended to be reusable building blocks for defining application environment characteristics. By creating custom Profiles, platform builders can curate the types of workloads that their platform supports, while platform users can reference them to understand the APIs that are available on the platform.

Spaces conceptual diagram showing Profiles added to previous diagram

Spaces include Profiles to describe the APIs and CRDs (in the form of Capabilities) available in the Space. Spaces transitively require all Capabilities that are required by any Profiles a Space includes. A Space will be scheduled only on a cluster that provides the required Capabilities of every Profile in by the Space.

For example, a platform builder that is knowledgeable on how to deploy Spring applications might require the “spring-cloud-gateway” and “service-binding” Capabilities in a Spring Profile. The Spring Profile provides a reusable way to create environments for running Spring applications. A Spring application team can easily use the Spring Profile to create a Space in a way that is company approved and guaranteed to be supported by the platform. The application team that needs a Gateway in front of their application, can use the Spring Cloud Gateway CRDs to create their own Gateway.

Profiles are intended to be reused across Spaces. In the example above, the team building the platform is attempting to standardize the way that their organization runs Spring applications. Therefore, every Spring application should be running in a Space with the same “spring” Profile. It is a common pattern for organizations to curate a limited set of Profiles that represent commonly used patterns and use cases.

Traits – Consistent and Reusable Functionality

Traits are collections of Kubernetes resources that are deployed into Spaces when they are created. They deliver required, pre-configured, and consistent content to Spaces.

Spaces conceptual diagram adding Traits to prior diagram

Profiles include Traits to easily create reusable content in Spaces when they are scheduled. This gives platform builders a way to simplify the configuration and creation of Kubernetes resources for their developers.

Continuing the example from above, the platform team notices that each team that uses the Spring Profile is re-implementing a way to create a Spring Cloud Gateway. They realize that this is not only inefficient, but also insecure as many teams are not sending their network traffic through the company’s central audit tool. To streamline developer flows and ensure the platform adheres to company security policy, a platform builder creates a Trait that contains a SpringCloudGateway resource with pre-configured connections to their auditing systems. They include the Trait in the Spring Profile that was previously created and used by various application teams. That change has now enabled every Space that uses the Spring Profile to use the newly created and company approved Spring Cloud Gateway definition.

Traits are not intended to be the primary way to require Capabilities or to deliver Kubernetes resources into Spaces. In most cases, the content, like application-specific code being delivered into a Space, is unique enough that it should be delivered directly to the Space via tanzu deploy. For example, while many applications may use a database, few of them will be connected to exactly the same database. In this case, the ServiceBinding for each database should be created and applied to the Space using tanzu service create, and not as a templated, reusable Trait. Most organizations will have very few Traits that represent a small number of truly identical deliverables.

check-circle-line exclamation-circle-line close-line
Scroll to top icon