Consuming AWS RDS on Tanzu Application Platform with Crossplane

Overview

This topic describes how to use Services Toolkit to enable Tanzu Application Platform workloads to consume AWS RDS PostgreSQL databases.

This topic makes use of Crossplane to manage RDS instances in AWS. It is an alternative approach to using the AWS Controllers for Kubernetes (ACK) to achieve the same outcomes.

Prerequisites

Meet these prerequisites:

  • Create a Kubernetes cluster that supports running both Tanzu Application Platform and Crossplane
  • Install Tanzu Application Platform on the Kubernetes cluster
  • Gain access to an AWS account with permissions to manage RDS database instances
  • Install AWS CLI
  • Configure a named profile for an AWS account that has permissions to manage RDS databases.

Install Crossplane

Run the following commands to install Crossplane to your existing Kubernetes cluster:

kubectl create namespace crossplane-system

helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

helm install crossplane --namespace crossplane-system crossplane-stable/crossplane \
  --set 'args={--enable-external-secret-stores}'

Note: For the latest steps for installing Crossplane, see the Crossplane documentation. As of Crossplane 1.9.0, the feature flag --enable-external-secret-stores is still needed.

For this topic, you do not need to install the Crossplane CLI or any additional configuration package.

Install AWS Provider for Crossplane

To install the AWS Provider for Crossplane:

  1. Run:

    kubectl apply -f -<<EOF
    ---
    apiVersion: pkg.crossplane.io/v1
    kind: Provider
    metadata:
     name: provider-aws
    spec:
     package: xpkg.upbound.io/crossplane/provider-aws:v0.24.1
    EOF
    
  2. After installing the provider, you see a new rdsinstances.database.aws.crossplane.io API resource available in your Kubernetes cluster. See the health of the installed provider by running:

    kubectl get provider.pkg.crossplane.io provider-aws
    

Configure AWS provider

To configure an AWS provider:

  1. Create a new key file:

    AWS_PROFILE=default && echo -e "[default]\naws_access_key_id = $(aws configure get aws_access_key_id --profile $AWS_PROFILE)\naws_secret_access_key = $(aws configure get aws_secret_access_key --profile $AWS_PROFILE)\naws_session_token = $(aws configure get aws_session_token --profile $AWS_PROFILE)" > creds.conf
    

    If your AWS profile is not named default, change AWS_PROFILE to the actual name.

  2. Verify that you a created a new key file by reading the content of the newly created creds.conf file.

  3. Create a new secret from the key file by running:

    kubectl create secret generic aws-provider-creds -n crossplane-system --from-file=creds=./creds.conf
    
  4. Delete the key file by running:

    rm -f creds.conf
    
  5. Configure the AWS provider to use the newly created secret by running:

    kubectl apply -f -<<EOF
    ---
    apiVersion: aws.crossplane.io/v1beta1
    kind: ProviderConfig
    metadata:
     name: default
    spec:
     credentials:
       source: Secret
       secretRef:
         namespace: crossplane-system
         name: aws-provider-creds
         key: creds
    EOF
    

Define composite resource types

Now that the AWS provider for Crossplane is installed and configured, you can create a new CompositeResourceDefinition (XRD) and corresponding Composition representing individual instances of RDS PostgreSQL by following the steps in this section. For more information about these concepts see the Crossplane composition documentation.

Instead of creating your own custom XRD and composition, you can also install an existing Crossplane configuration package for AWS that includes pre-configured XRDs and compositions for RDS.

The primary reason for choosing to create a new XRD and composition is to ensure the connection secrets for newly provisioned RDS PostgreSQL instances support the Service Binding Specification for Kubernetes and automatic Spring Boot configuration using Spring Cloud Bindings.

  1. Create a new XRD by running:

    kubectl apply -f -<<EOF
    ---
    apiVersion: apiextensions.crossplane.io/v1
    kind: CompositeResourceDefinition
    metadata:
     name: xpostgresqlinstances.bindable.database.example.org
    spec:
     claimNames:
       kind: PostgreSQLInstance
       plural: postgresqlinstances
     connectionSecretKeys:
     - type
     - provider
     - host
     - port
     - database
     - username
     - password
     group: bindable.database.example.org
     names:
       kind: XPostgreSQLInstance
       plural: xpostgresqlinstances
     versions:
     - name: v1alpha1
       referenceable: true
       schema:
         openAPIV3Schema:
           properties:
             spec:
               properties:
                 parameters:
                   properties:
                     storageGB:
                       type: integer
                   required:
                   - storageGB
                   type: object
               required:
               - parameters
               type: object
           type: object
       served: true
    EOF
    

    After the newly created XRD is reconciled there are two new API resources available in your Kubernetes cluster, xpostgresqlinstances.bindable.database.example.org and postgresqlinstances.bindable.database.example.org.

  2. Create a corresponding composition by running:

    kubectl apply -f -<<EOF
    ---
    apiVersion: apiextensions.crossplane.io/v1
    kind: Composition
    metadata:
     labels:
       provider: "aws"
       vpc: "default"
     name: xpostgresqlinstances.bindable.aws.database.example.org
    spec:
     compositeTypeRef:
       apiVersion: bindable.database.example.org/v1alpha1
       kind: XPostgreSQLInstance
     publishConnectionDetailsWithStoreConfigRef:
       name: default
     resources:
     - base:
         apiVersion: database.aws.crossplane.io/v1beta1
         kind: RDSInstance
         spec:
           forProvider:
             dbInstanceClass: db.t2.micro
             engine: postgres
             dbName: postgres
             engineVersion: "12"
             masterUsername: masteruser
             publiclyAccessible: true
             region: us-east-1
             skipFinalSnapshotBeforeDeletion: true
           writeConnectionSecretToRef:
             namespace: crossplane-system
       connectionDetails:
       - name: type
         value: postgresql
       - name: provider
         value: aws
       - name: database
         value: postgres
       - fromConnectionSecretKey: username
       - fromConnectionSecretKey: password
       - name: host
         fromConnectionSecretKey: endpoint
       - fromConnectionSecretKey: port
       name: rdsinstance
       patches:
       - fromFieldPath: metadata.uid
         toFieldPath: spec.writeConnectionSecretToRef.name
         transforms:
         - string:
             fmt: '%s-postgresql'
             type: Format
           type: string
         type: FromCompositeFieldPath
       - fromFieldPath: spec.parameters.storageGB
         toFieldPath: spec.forProvider.allocatedStorage
         type: FromCompositeFieldPath
    EOF
    

    This composition ensures that all RDS PostgreSQL instances are placed in the us-east-1 region and use the default VPC for the respective AWS account.

  3. Take one of these actions:

    • Connect to those instances from outside the default VPC by assigning an appropriate inbound rule for TCP on port 5432 to the security group of that VPC.
    • Define a composition that creates a separate VPC for each RDS PostgreSQL instance and automatically configures inbound rules. See this example.

Create an instance class

To make instances of a service easy for application operators to discover and claim, the service operator persona creates a ClusterInstanceClass. In this example, the class states that claimable instances of RDS PostgreSQL are represented by secret objects of type connection.crossplane.io/v1alpha1 with the label services.apps.tanzu.vmware.com/class set to rds-postgres:

kubectl apply -f -<<EOF
---
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
  name: rds-postgres
spec:
  description:
    short: AWS RDS Postgresql database instances
  pool:
    kind: Secret
    labelSelector:
      matchLabels:
        services.apps.tanzu.vmware.com/class: rds-postgres
    fieldSelector: type=connection.crossplane.io/v1alpha1
EOF

In addition, grant RBAC permissions to Services Toolkit to enable reading the secrets specified by the class.

kubectl apply -f -<<EOF
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: stk-secret-reader
  labels:
    servicebinding.io/controller: "true"
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
EOF

Provision RDS PostgreSQL instance

As the service operator persona, you now provision an instance of RDS PostgreSQL using the postgresqlinstances.bindable.database.example.org API managed by the XRD you previously created.

.spec.publishConnectionDetailsTo provides Crossplane with the name and a label for the secret that stores the connection details for the newly created database. You can see that the label specified here matches the drop-down menu value defined in the ClusterInstanceClass you created earlier.

  1. Create an RDS database instance in your AWS account by running:

    kubectl apply -f -<<EOF
    ---
    apiVersion: bindable.database.example.org/v1alpha1
    kind: PostgreSQLInstance
    metadata:
     name: rds-postgres-db
     namespace: default
    spec:
     parameters:
       storageGB: 20
     compositionSelector:
       matchLabels:
         provider: aws
         vpc: default
     publishConnectionDetailsTo:
       name: rds-postgres-db
       metadata:
         labels:
           services.apps.tanzu.vmware.com/class: rds-postgres
    EOF
    
  2. Verify that you created the RDS database instance by running:

    aws rds describe-db-instances --region us-east-1 --profile default
    

    Expect the status of the newly created PostgreSQLInstance resource to be READY=True. This might take a few minutes. You can wait for this by running:

    kubectl wait --for=condition=Ready=true postgresqlinstances.bindable.database.example.org rds-postgres-db
    

As soon as the RDS PostgreSQL instance is ready, it is claimable by the application operator persona as shown in the next section.

Claim the RDS PostgreSQL instance and connect to it from the Tanzu Application Platform workload

Thanks to the ClusterInstanceClass created in the earlier section, application operators can now use the Tanzu CLI to discover and claim secrets representing RDS PostgreSQL instances.

  1. Show available classes of service instances by running:

    tanzu service classes list
    
    NAME            DESCRIPTION
    rds-postgres    AWS RDS Postgresql database instances
    
  2. Show claimable instances belonging to the RDS PostgreSQL class by running:

    tanzu services claimable list --class rds-postgres
    
    NAME               NAMESPACE  API KIND  API GROUP/VERSION
    rds-postgres-db    default    Secret    v1
    
  3. Create a claim for the discovered secret by running:

    tanzu service claim create rds-claim \
    --resource-name rds-postgres-db \
    --resource-kind Secret \
    --resource-api-version v1
    
  4. Obtain the claim reference by running:

    tanzu service claim list -o wide
    

    Expect to see the following output:

    NAME                     READY  REASON  CLAIM REF
    rds-claim                True           services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:rds-claim
    
  5. Create an application workload that consumes the claimed RDS PostgreSQL database. In this example, --service-ref is set to the claim reference obtained earlier.

    tanzu apps workload create my-workload \
    --git-repo https://github.com/sample-accelerators/spring-petclinic \
    --git-branch main \
    --git-tag tap-1.2 \
    --type web \
    --label app.kubernetes.io/part-of=spring-petclinic \
    --annotation autoscaling.knative.dev/minScale=1 \
    --env SPRING_PROFILES_ACTIVE=postgres \
    --service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:rds-claim
    
check-circle-line exclamation-circle-line close-line
Scroll to top icon