Consuming AWS RDS on Tanzu Application Platform with AWS Controllers for Kubernetes (ACK)

This topic describes how to use Services Toolkit to allow Tanzu Application Platform workloads to consume AWS RDS PostgreSQL databases.

This topic makes use of AWS Controllers for Kubernetes (ACK) to manage RDS instances in AWS. As such, it is an alternative approach to using Crossplane to achieve the same outcomes.

Prerequisites

Create service instances that are compatible with Tanzu Application Platform

Installing the ACK service controller for RDS makes available new Kubernetes APIs for interacting with RDS resources from within the Tanzu Application Platform cluster.

$ kubectl api-resources --api-group rds.services.k8s.aws

NAME                       SHORTNAMES   APIVERSION                      NAMESPACED   KIND
dbclusterparametergroups                rds.services.k8s.aws/v1alpha1   true         DBClusterParameterGroup
dbclusters                              rds.services.k8s.aws/v1alpha1   true         DBCluster
dbinstances                             rds.services.k8s.aws/v1alpha1   true         DBInstance
dbparametergroups                       rds.services.k8s.aws/v1alpha1   true         DBParameterGroup
dbsubnetgroups                          rds.services.k8s.aws/v1alpha1   true         DBSubnetGroup
globalclusters                          rds.services.k8s.aws/v1alpha1   true         GlobalCluster

DBInstance is of most interest here because this is the primary API for creating RDS databases. However, there are two important obstacles with this API when considering compatibility with Tanzu Application Platform.

Obstacle 1: DBInstance does not adhere to the binding specification

DBInstance does not adhere to the Service Binding Specification for Kubernetes. Tanzu Application Platform uses this specification as a contract for ensuring compatibility between different parts of the system. Given that DBInstance does not adhere to the specification it means that, by default, it is not possible to claim and bind application workloads to DBInstance resources.

Obstacle 2: Creating a DBInstance resource on its own is not sufficient

Creating a DBInstance resource on its own might not always be enough to create a working, usable instance that can be connected to and utilized.

For example, DBInstance defines the field .spec.masterUserPassword, which must refer to a secret containing credentials for the instance. As such, the secret resource can be considered a dependent resource of DBInstance. Without both of these resources, it is not possible to properly configure the RDS instance as wanted. In many cases, a group of related resources must be created to create something usable.

Solutions

Tanzu Application Platform v1.2 and later enables solutions for both these obstacles.

For example, consider the first obstacle where DBInstance does not adhere to the Kubernetes binding specification. One solution is for the authors of the RDS ACK service controller to update the DBInstance API to make it adhere to the binding specification. However, this requires code changes to the operator itself, and the authors of the operator might choose not to prioritize it.

Fortunately, there is an alternative solution that doesn’t require any code changes to the operator itself while still enabling claiming and binding to RDS instances from within a Tanzu Application Platform cluster.

This solution uses the SecretTemplate API provided by Carvel’s secretgen-controller. This API can be used to create binding specification-conforming secrets by identifying and collecting information that resources from the RDS APIs provide.

Next, consider the second obstacle where multiple resources must be created to produce a usable RDS database. One solution to this obstacle is to just document all the resources that must be created to produce something that can be used. This solution is laborious, error-prone, and is generally a poor developer experience.

Fortunately, there is an alternative solution that abstracts away the complexities of creating instances that are known to work well with application workloads.

This solution uses the ClusterInstanceClass API provided by Services Toolkit. Instance classes allow for logical service instances to be presented to Application Operators, allowing them to discover, reason about, and, most importantly, claim service instances that they can then bind to their application workloads.

The rest of this topic describes how both these solutions can come together to form an end-to-end integration for RDS services on Tanzu Application Platform.

Create an RDS service instance

This section describes how to create an RDS service instance in Tanzu Application Platform by using a ready-made reference Carvel Package. This step is typically performed by the Service Operator role. Follow the steps in Creating an RDS service instance by using a Carvel Package.

Alternatively, if you want to author your own reference package and want to learn about the underlying APIs and how they come together to produce a useable service instance for Tanzu Application Platform, you can achieve the same outcome by using the more advanced Creating an RDS service instance manually.

After you complete either of these steps and have a running RDS service instance, return here to continue with the rest of the use case.

Create a service instance class for RDS

Now that you know how to create RDS service instances it’s time to learn how to make those instances discoverable to Application Operators. This step is typically performed by the Service Operator role.

You can use Services Toolkit’s ClusterInstanceClass API to create a service instance class to represent RDS service instances within the cluster. The existence of such classes make these logical service instances discoverable to Application Operators. This allows them to create Resource Claims for such instances and to then bind them to application workloads.

Create the following Kubernetes resource on your EKS cluster:

# clusterinstanceclass.yaml
---
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
  name: aws-rds-postgres
spec:
  description:
    short: AWS RDS instances with a postgresql engine
  pool:
    kind: Secret
    labelSelector:
      matchLabels:
        services.apps.tanzu.vmware.com/class: rds-postgres

Apply it by running:

kubectl apply -f clusterinstanceclass.yaml

In this example, the class states that claimable instances of RDS PostgreSQL are represented by Secret objects with the label services.apps.tanzu.vmware.com/class set to rds-postgres. A Secret with this label was created in the earlier step when you provisioned an RDS service instance.

Although this example uses services.apps.tanzu.vmware.com/class, there is no special meaning to that key. The Service Operator role can choose arbitrary label names and values. They might also decide to select multiple labels or combine a label selector with a field selector when defining the ClusterInstanceClass.

After creating a ClusterInstanceClass, you must grant sufficient RBAC permissions to enable Services Toolkit to read the resources that match the pool definition of the instance class. For this example, create the following aggregated ClusterRole in your EKS cluster:

# stk-secret-reader.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: stk-secret-reader
  labels:
    servicebinding.io/controller: "true"
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch

Apply it by running:

kubectl apply -f stk-secret-reader.yaml

If you want to claim resources across namespace boundaries, you must create a corresponding ResourceClaimPolicy. For example, if the provisioned RDS PostgreSQL instances exist in the namespace service-instances, and you want to allow Application Operators to claim them for workloads residing in the default namespace, create the following ResourceClaimPolicy:

# resourceclaimpolicy.yaml
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ResourceClaimPolicy
metadata:
  name: default-can-claim-rds-postgres
  namespace: service-instances
spec:
  subject:
    kind: Secret
    group: ""
    selector:
      matchLabels:
        services.apps.tanzu.vmware.com/class: rds-postgres
  consumingNamespaces: [ "default" ]

Apply it by running:

kubectl apply -f resourceclaimpolicy.yaml

Discover, Claim, and Bind to an RDS

Creating the ClusterInstanceClass and the corresponding RBAC informs Application Operators that RDS is available to use with their application workloads on Tanzu Application Platform. In this section you learn how to discover, claim, and bind to the RDS service instance previously created. The Application Operator is typically the role that discovers and claims service instances. The Application Developer is typically the role that handles binding.

To discover what service instances are available to them, Application Operators can run

tanzu services classes list

  NAME                  DESCRIPTION
  aws-rds-postgres      AWS RDS instances with a postgresql engine

Here you can see information about the ClusterInstanceClass created in the earlier step. Each ClusterInstanceClass created is added to the list of classes returned here.

The next step is to claim an instance of the wanted class, but to do that, Application Operators must first discover the list of currently claimable instances for the class. Many variables, including namespace boundaries, claim policies, and the exclusivity of claims, affect the capacity to claim instances. Therefore Services Toolkit provides the CLI command tanzu service claimable list to help inform Application Operators of the instances that can enable successful claims. Example:

tanzu services claimable list --class aws-rds-postgres

  NAME          NAMESPACE  API KIND  API GROUP/VERSION
  rds-bindable  default    Secret    v1

Because of the setup performed as part of Creating a claimable class for RDS instances, the secrets created from the SecretTemplate as part of Create an RDS service instance now appear as claimable to the Application Operator. From here on it is simply a case of creating a resource claim for the instance and then binding the claim to an application workload.

Create a claim for the newly created secret by running:

tanzu service claim create ack-rds-claim \
  --resource-name rds-bindable \
  --resource-kind Secret \
  --resource-api-version v1

Obtain the claim reference of the claim by running:

tanzu service claim list -o wide

Verify that the output is similar to the following:

NAME                    READY  REASON  CLAIM REF
ack-rds-claim           True           services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:ack-rds-claim

Create an application workload that consumes the claimed RDS PostgreSQL Database. Example:

tanzu apps workload create my-workload \
  --git-repo https://github.com/sample-accelerators/spring-petclinic \
  --git-branch main \
  --git-tag tap-1.2 \
  --type web \
  --label app.kubernetes.io/part-of=spring-petclinic \
  --annotation autoscaling.knative.dev/minScale=1 \
  --env SPRING_PROFILES_ACTIVE=postgres \
  --service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:ack-rds-claim

--service-ref is set to the claim reference obtained previously.

Your application workload now starts up and connects automatically to the RDS service instance. You can verify this by visiting the app in the browser and, for example, creating a new owner through the UI.

check-circle-line exclamation-circle-line close-line
Scroll to top icon