This topic describes how to use Services Toolkit to allow Tanzu Application Platform workloads to consume AWS RDS PostgreSQL databases.
This topic makes use of AWS Controllers for Kubernetes (ACK) to manage RDS instances in AWS. As such, it is an alternative approach to using Crossplane to achieve the same outcomes.
Installing the ACK service controller for RDS makes available new Kubernetes APIs for interacting with RDS resources from within the Tanzu Application Platform cluster.
$ kubectl api-resources --api-group rds.services.k8s.aws
NAME SHORTNAMES APIVERSION NAMESPACED KIND
dbclusterparametergroups rds.services.k8s.aws/v1alpha1 true DBClusterParameterGroup
dbclusters rds.services.k8s.aws/v1alpha1 true DBCluster
dbinstances rds.services.k8s.aws/v1alpha1 true DBInstance
dbparametergroups rds.services.k8s.aws/v1alpha1 true DBParameterGroup
dbsubnetgroups rds.services.k8s.aws/v1alpha1 true DBSubnetGroup
globalclusters rds.services.k8s.aws/v1alpha1 true GlobalCluster
DBInstance is of most interest here because this is the primary API for creating RDS databases. However, there are two important obstacles with this API when considering compatibility with Tanzu Application Platform.
DBInstance does not adhere to the binding specificationDBInstance does not adhere to the Service Binding Specification for Kubernetes. Tanzu Application Platform uses this specification as a contract for ensuring compatibility between different parts of the system. Given that DBInstance does not adhere to the specification it means that, by default, it is not possible to claim and bind application workloads to DBInstance resources.
DBInstance resource on its own is not sufficientCreating a DBInstance resource on its own might not always be enough to create a working, usable instance that can be connected to and utilized.
For example, DBInstance defines the field .spec.masterUserPassword, which must refer to a secret containing credentials for the instance. As such, the secret resource can be considered a dependent resource of DBInstance. Without both of these resources, it is not possible to properly configure the RDS instance as wanted. In many cases, a group of related resources must be created to create something usable.
Tanzu Application Platform v1.2 and later enables solutions for both these obstacles.
For example, consider the first obstacle where DBInstance does not adhere to the Kubernetes binding specification. One solution is for the authors of the RDS ACK service controller to update the DBInstance API to make it adhere to the binding specification. However, this requires code changes to the operator itself, and the authors of the operator might choose not to prioritize it.
Fortunately, there is an alternative solution that doesn’t require any code changes to the operator itself while still enabling claiming and binding to RDS instances from within a Tanzu Application Platform cluster.
This solution uses the SecretTemplate API provided by Carvel’s secretgen-controller. This API can be used to create binding specification-conforming secrets by identifying and collecting information that resources from the RDS APIs provide.
Next, consider the second obstacle where multiple resources must be created to produce a usable RDS database. One solution to this obstacle is to just document all the resources that must be created to produce something that can be used. This solution is laborious, error-prone, and is generally a poor developer experience.
Fortunately, there is an alternative solution that abstracts away the complexities of creating instances that are known to work well with application workloads.
This solution uses the ClusterInstanceClass API provided by Services Toolkit. Instance classes allow for logical service instances to be presented to Application Operators, allowing them to discover, reason about, and, most importantly, claim service instances that they can then bind to their application workloads.
The rest of this topic describes how both these solutions can come together to form an end-to-end integration for RDS services on Tanzu Application Platform.
This section describes how to create an RDS service instance in Tanzu Application Platform by using a ready-made reference Carvel Package. This step is typically performed by the Service Operator role. Follow the steps in Creating an RDS service instance by using a Carvel Package.
Alternatively, if you want to author your own reference package and want to learn about the underlying APIs and how they come together to produce a useable service instance for Tanzu Application Platform, you can achieve the same outcome by using the more advanced Creating an RDS service instance manually.
After you complete either of these steps and have a running RDS service instance, return here to continue with the rest of the use case.
Now that you have created an RDS service instance, you need to grant sufficient RBAC permissions to enable Services Toolkit to read its credentials. For this example, create the following aggregated ClusterRole in your EKS cluster:
# stk-secret-reader.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: stk-secret-reader
labels:
servicebinding.io/controller: "true"
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
kubectl apply -f stk-secret-reader.yaml
Now that you know how to create RDS service instances it’s time to learn how to make those instances discoverable to Application Operators. This step is typically performed by the Service Operator role.
You can use Services Toolkit’s ClusterInstanceClass API to create a service instance class to represent RDS service instances within the cluster. The existence of such classes make these logical service instances discoverable to Application Operators. This allows them to create Resource Claims for such instances and to then bind them to application workloads.
Create the following Kubernetes resource on your EKS cluster:
# clusterinstanceclass.yaml
---
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
name: aws-rds-postgres
spec:
description:
short: AWS RDS instances with a postgresql engine
pool:
kind: Secret
labelSelector:
matchLabels:
services.apps.tanzu.vmware.com/class: rds-postgres
Apply it by running:
kubectl apply -f clusterinstanceclass.yaml
In this example, the class states that claimable instances of RDS PostgreSQL are represented by Secret objects with the label services.apps.tanzu.vmware.com/class set to rds-postgres. A Secret with this label was created in the earlier step when you provisioned an RDS service instance.
Although this example uses services.apps.tanzu.vmware.com/class, there is no special meaning to that key. The Service Operator role can choose arbitrary label names and values. They might also decide to select multiple labels or combine a label selector with a field selector when defining the ClusterInstanceClass.
If you want to claim resources across namespace boundaries, you must create a corresponding ResourceClaimPolicy. For example, if the provisioned RDS PostgreSQL instances exist in the namespace service-instances, and you want to allow Application Operators to claim them for workloads residing in the default namespace, create the following ResourceClaimPolicy:
# resourceclaimpolicy.yaml
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ResourceClaimPolicy
metadata:
name: default-can-claim-rds-postgres
namespace: service-instances
spec:
subject:
kind: Secret
group: ""
selector:
matchLabels:
services.apps.tanzu.vmware.com/class: rds-postgres
consumingNamespaces: [ "default" ]
Apply it by running:
kubectl apply -f resourceclaimpolicy.yaml
Creating the ClusterInstanceClass and the corresponding RBAC informs Application Operators that RDS is available to use with their application workloads on Tanzu Application Platform. In this section you learn how to discover, claim, and bind to the RDS service instance previously created. The Application Operator is typically the role that discovers and claims service instances. The Application Developer is typically the role that handles binding.
To discover what service instances are available to them, Application Operators can run
tanzu services classes list
NAME DESCRIPTION
aws-rds-postgres AWS RDS instances with a postgresql engine
Here you can see information about the ClusterInstanceClass created in the earlier step. Each ClusterInstanceClass created is added to the list of classes returned here.
The next step is to claim an instance of the wanted class, but to do that, Application Operators must first discover the list of currently claimable instances for the class. Many variables, including namespace boundaries, claim policies, and the exclusivity of claims, affect the capacity to claim instances. Therefore Services Toolkit provides the CLI command tanzu service claimable list to help inform Application Operators of the instances that can enable successful claims. Example:
tanzu services claimable list --class aws-rds-postgres
NAME NAMESPACE API KIND API GROUP/VERSION
rds-bindable default Secret v1
Because of the setup performed as part of Creating a claimable class for RDS instances, the secrets created from the SecretTemplate as part of Create an RDS service instance now appear as claimable to the Application Operator. From here on it is simply a case of creating a resource claim for the instance and then binding the claim to an application workload.
Create a claim for the newly created secret by running:
tanzu service resource-claim create ack-rds-claim \
--resource-name rds-bindable \
--resource-kind Secret \
--resource-api-version v1
Obtain the claim reference of the claim by running:
tanzu service resource-claim list -o wide
Verify that the output is similar to the following:
NAME READY REASON CLAIM REF
ack-rds-claim True services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:ack-rds-claim
Create an application workload that consumes the claimed RDS PostgreSQL Database. Example:
tanzu apps workload create my-workload \
--git-repo https://github.com/sample-accelerators/spring-petclinic \
--git-branch main \
--git-tag tap-1.2 \
--type web \
--label app.kubernetes.io/part-of=spring-petclinic \
--annotation autoscaling.knative.dev/minScale=1 \
--env SPRING_PROFILES_ACTIVE=postgres \
--service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:ack-rds-claim
--service-ref is set to the claim reference obtained previously.
Your application workload now starts up and connects automatically to the RDS service instance. You can verify this by visiting the app in the browser and, for example, creating a new owner through the UI.
