This topic describes how to use Services Toolkit to enable Tanzu Application Platform workloads to consume AWS RDS PostgreSQL databases.
This topic makes use of Crossplane to manage RDS instances in AWS. It is an alternative approach to using the AWS Controllers for Kubernetes (ACK) to achieve the same outcomes.
Note: This usecase is not currently compatible with TAP air-gapped installations.
Meet these prerequisites:
Note: For the latest steps for installing Crossplane, see these instructions. For the instructions in this topic, it is important to enable support for external secret stores in Crossplane. This is currently an Alpha feature. As such, you will have to explicitly set command line flag
--enable-external-secret-storeswhen starting the Crossplane controller.
Run the following commands to install Crossplane to your existing Kubernetes cluster:
kubectl create namespace crossplane-system
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update
helm install crossplane --namespace crossplane-system crossplane-stable/crossplane \
--set 'args={--enable-external-secret-stores}'
For this topic, you do not need to install the Crossplane CLI or any additional configuration package.
To install the AWS Provider for Crossplane:
Run:
kubectl apply -f -<<EOF
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-aws
spec:
package: xpkg.upbound.io/crossplane-contrib/provider-aws:v0.33.0
EOF
After installing the provider, you see a new rdsinstances.database.aws.crossplane.io API resource available in your Kubernetes cluster. See the health of the installed provider by running:
kubectl get provider.pkg.crossplane.io provider-aws
To configure an AWS provider:
Ensure you are logged into using the aws cli and can view db instances.
AWS_PROFILE=default && aws rds describe-db-instances --region us-east-1 --profile $AWS_PROFILE
If your AWS profile is not named default, change AWS_PROFILE to the actual name.
Create a new key file:
AWS_PROFILE=default && echo -e "[default]\naws_access_key_id = $(aws configure get aws_access_key_id --profile $AWS_PROFILE)\naws_secret_access_key = $(aws configure get aws_secret_access_key --profile $AWS_PROFILE)\naws_session_token = $(aws configure get aws_session_token --profile $AWS_PROFILE)" > creds.conf
If your AWS profile is not named default, change AWS_PROFILE to the actual name.
Verify that you a created a new key file by reading the content of the newly created creds.conf file.
Create a new secret from the key file by running:
kubectl create secret generic aws-provider-creds -n crossplane-system --from-file=creds=./creds.conf
Delete the key file by running:
rm -f creds.conf
Configure the AWS provider to use the newly created secret by running:
kubectl apply -f -<<EOF
---
apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: aws-provider-creds
key: creds
EOF
Now that the AWS provider for Crossplane is installed and configured, you can create a new CompositeResourceDefinition (XRD) and corresponding Composition representing individual instances of RDS PostgreSQL by following the steps in this section. For more information about these concepts see the Crossplane composition documentation.
Instead of creating your own custom XRD and composition, you can also install an existing Crossplane configuration package for AWS that includes pre-configured XRDs and compositions for RDS.
The primary reason for choosing to create a new XRD and composition is to ensure the connection secrets for newly provisioned RDS PostgreSQL instances support the Service Binding Specification for Kubernetes and automatic Spring Boot configuration using Spring Cloud Bindings.
Create a new XRD by running:
kubectl apply -f -<<EOF
---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xpostgresqlinstances.bindable.database.example.org
spec:
claimNames:
kind: PostgreSQLInstance
plural: postgresqlinstances
connectionSecretKeys:
- type
- provider
- host
- port
- database
- username
- password
group: bindable.database.example.org
names:
kind: XPostgreSQLInstance
plural: xpostgresqlinstances
versions:
- name: v1alpha1
referenceable: true
schema:
openAPIV3Schema:
properties:
spec:
properties:
parameters:
properties:
storageGB:
type: integer
required:
- storageGB
type: object
required:
- parameters
type: object
type: object
served: true
EOF
After the newly created XRD is reconciled there are two new API resources available in your Kubernetes cluster, xpostgresqlinstances.bindable.database.example.org and postgresqlinstances.bindable.database.example.org.
Create a corresponding composition by running:
kubectl apply -f -<<EOF
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
labels:
provider: "aws"
vpc: "default"
name: xpostgresqlinstances.bindable.aws.database.example.org
spec:
compositeTypeRef:
apiVersion: bindable.database.example.org/v1alpha1
kind: XPostgreSQLInstance
publishConnectionDetailsWithStoreConfigRef:
name: default
resources:
- base:
apiVersion: database.aws.crossplane.io/v1beta1
kind: RDSInstance
spec:
forProvider:
dbInstanceClass: db.t2.micro
engine: postgres
dbName: postgres
engineVersion: "12"
masterUsername: masteruser
publiclyAccessible: true
region: us-east-1
skipFinalSnapshotBeforeDeletion: true
writeConnectionSecretToRef:
namespace: crossplane-system
connectionDetails:
- name: type
value: postgresql
- name: provider
value: aws
- name: database
value: postgres
- fromConnectionSecretKey: username
- fromConnectionSecretKey: password
- name: host
fromConnectionSecretKey: endpoint
- fromConnectionSecretKey: port
name: rdsinstance
patches:
- fromFieldPath: metadata.uid
toFieldPath: spec.writeConnectionSecretToRef.name
transforms:
- string:
fmt: '%s-postgresql'
type: Format
type: string
type: FromCompositeFieldPath
- fromFieldPath: spec.parameters.storageGB
toFieldPath: spec.forProvider.allocatedStorage
type: FromCompositeFieldPath
EOF
This composition ensures that all RDS PostgreSQL instances are placed in the us-east-1 region and use the default VPC for the respective AWS account.
Take one of these actions:
5432 to the security group of that VPC.As the service operator persona, you now provision an instance of RDS PostgreSQL using the postgresqlinstances.bindable.database.example.org API managed by the XRD you previously created.
.spec.publishConnectionDetailsTo provides Crossplane with the name and a label for the secret that stores the connection details for the newly created database.
Create an RDS database instance in your AWS account by running:
kubectl apply -f -<<EOF
---
apiVersion: bindable.database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
name: rds-postgres-db
namespace: default
spec:
parameters:
storageGB: 20
compositionSelector:
matchLabels:
provider: aws
vpc: default
publishConnectionDetailsTo:
name: rds-postgres-db
metadata:
labels:
services.apps.tanzu.vmware.com/class: rds-postgres
EOF
Verify that you created the RDS database instance by running:
aws rds describe-db-instances --region us-east-1 --profile default
Expect the status of the newly created PostgreSQLInstance resource to be READY=True. This might take a few minutes. You can wait for this by running:
kubectl wait --for=condition=Ready=true postgresqlinstances.bindable.database.example.org rds-postgres-db
As soon as the RDS PostgreSQL instance is ready, it is claimable by the application operator persona as shown in the next sections.
To make instances of a service easy for application operators to discover and claim, the service operator persona creates a ClusterInstanceClass. In this example, the class states that claimable instances of RDS PostgreSQL are represented by secret objects of type connection.crossplane.io/v1alpha1 with the label services.apps.tanzu.vmware.com/class set to rds-postgres:
kubectl apply -f -<<EOF
---
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
name: rds-postgres
spec:
description:
short: AWS RDS Postgresql database instances
pool:
kind: Secret
labelSelector:
matchLabels:
services.apps.tanzu.vmware.com/class: rds-postgres
fieldSelector: type=connection.crossplane.io/v1alpha1
EOF
You can see that the label specified here matches the label defined in the RDS instance created previously.
In addition, grant RBAC permissions to Services Toolkit to enable reading the secrets specified by the class.
kubectl apply -f -<<EOF
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: stk-secret-reader
labels:
servicebinding.io/controller: "true"
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
EOF
Thanks to the ClusterInstanceClass created in the earlier section, application operators can now use the Tanzu CLI to discover and claim secrets representing RDS PostgreSQL instances.
Show available classes of service instances by running:
tanzu service classes list
NAME DESCRIPTION
rds-postgres AWS RDS Postgresql database instances
Show claimable instances belonging to the RDS PostgreSQL class by running:
tanzu services claimable list --class rds-postgres
NAME NAMESPACE API KIND API GROUP/VERSION
rds-postgres-db default Secret v1
Create a claim for the discovered secret by running:
tanzu service resource-claim create rds-claim \
--resource-name rds-postgres-db \
--resource-kind Secret \
--resource-api-version v1
Obtain the claim reference by running:
tanzu service resource-claim list -o wide
Expect to see the following output:
NAME READY REASON CLAIM REF
rds-claim True services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:rds-claim
Create an application workload that consumes the claimed RDS PostgreSQL database. In this example, --service-ref is set to the claim reference obtained earlier.
tanzu apps workload create my-workload \
--git-repo https://github.com/sample-accelerators/spring-petclinic \
--git-branch main \
--git-tag tap-1.2 \
--type web \
--label app.kubernetes.io/part-of=spring-petclinic \
--annotation autoscaling.knative.dev/minScale=1 \
--env SPRING_PROFILES_ACTIVE=postgres \
--service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:rds-claim
