This topic demonstrates how to use Services Toolkit to allow Tanzu Application Platform workloads to consume GCP CloudSQL PostgreSQL databases. This particular guide makes use of Crossplane to manage CloudSQL instances in GCP. As such, it can be thought of as an alternative approach to Consuming Google Cloud SQL on Tanzu Application Platform (TAP) with Config Connector to achieve the same outcomes.
Note This usecase is not currently compatible with TAP air-gapped installations.
Meet these prerequisites:
Note: For the latest steps for installing Crossplane, see these instructions. For the instructions in this topic, it is important to enable support for external secret stores in Crossplane. This is currently an Alpha feature. As such, you will have to explicitly set command line flag
--enable-external-secret-storeswhen starting the Crossplane controller.
Run the following commands to install Crossplane to your existing Kubernetes cluster:
kubectl create namespace crossplane-system
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update
helm install crossplane --namespace crossplane-system crossplane-stable/crossplane \
--set 'args={--enable-external-secret-stores}'
For this topic, you do not need to install the Crossplane CLI or any additional configuration package.
To install the GCP Provider for Crossplane, run:
kubectl apply -f -<<EOF
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: crossplane-provider-gcp
spec:
package: crossplane/provider-gcp:v0.21.0
EOF
After you have installed the provider, you see a new cloudsqlinstances.database.gcp.crossplane.io API resource available in your Kubernetes cluster. See the health of the installed provider by running:
kubectl get provider.pkg.crossplane.io crossplane-provider-gcp
This section creates a new GCP Service Account and gives it permissions to manage CloudSQL databases which are necessary to use Crossplane to manage CloudSQL instances.
Create a new GCP ServiceAccount, give it Cloud SQL Admin and create a key file:
PROJECT_ID=<GCP Project ID>
SA_NAME=crossplane-cloudsql
gcloud iam service-accounts create "${SA_NAME}" --project "${PROJECT_ID}"
gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
--role="roles/cloudsql.admin" \
--member "serviceAccount:${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
gcloud iam service-accounts keys create creds.json --project "${PROJECT_ID}" --iam-account "${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
Create a new secret from the key file by running:
kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./creds.json
Delete the key file by running:
rm -f creds.json
Configure the GCP provider to use the newly created secret by running:
kubectl apply -f -<<EOF
apiVersion: gcp.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
projectID: ${PROJECT_ID}
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: gcp-creds
key: creds
EOF
Now that the GCP provider for Crossplane has been installed and configured, create a new CompositeResourceDefinition (XRD) and corresponding Composition representing individual instances of CloudSQL Postgresql. For more information about these concepts see the Crossplane Composition documentation.
Note: Instead of creating your own custom XRD and Composition as shown below, you can also install an existing Crossplane configuration package for GCP that includes pre-configured XRDs and compositions for CloudSQL. The primary reason for creating a new XRD and composition from scratch is to make sure the connection secrets for newly provisioned CloudSQL Postgresql instances support the Service Binding Specification for Kubernetes and automatic Spring Boot configuration using Spring Cloud Bindings.
Create a new XRD by running:
kubectl apply -f -<<EOF
---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xpostgresqlinstances.bindable.database.example.org
spec:
claimNames:
kind: PostgreSQLInstance
plural: postgresqlinstances
connectionSecretKeys:
- type
- provider
- host
- port
- database
- username
- password
group: bindable.database.example.org
names:
kind: XPostgreSQLInstance
plural: xpostgresqlinstances
versions:
- name: v1alpha1
referenceable: true
schema:
openAPIV3Schema:
properties:
spec:
properties:
parameters:
properties:
storageGB:
type: integer
required:
- storageGB
type: object
required:
- parameters
type: object
type: object
served: true
EOF
After the newly created XRD has been successfully reconciled, there are two new API resources available in your Kubernetes cluster, xpostgresqlinstances.bindable.database.example.org and postgresqlinstances.bindable.database.example.org. The XRD created is agnostic to the underlying cloud managed service, so could also be fulfilled by a Composition that makes use of AWS RDS Postgresql or Azure Database for PostgreSQL.
Create a corresponding composition (not in a production environment) by running:
kubectl apply -f -<<EOF
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
labels:
provider: gcp
name: xpostgresqlinstances.bindable.gcp.database.example.org
spec:
compositeTypeRef:
apiVersion: bindable.database.example.org/v1alpha1
kind: XPostgreSQLInstance
publishConnectionDetailsWithStoreConfigRef:
name: default
resources:
- base:
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
spec:
forProvider:
databaseVersion: POSTGRES_14
region: us-central1
settings:
dataDiskType: PD_SSD
ipConfiguration:
authorizedNetworks:
- value: 0.0.0.0/0 # not recommended for production deployments!
ipv4Enabled: true
tier: db-custom-1-3840
writeConnectionSecretToRef:
namespace: crossplane-system
connectionDetails:
- name: type
value: postgresql
- name: provider
value: gcp
- name: database
value: postgres
- fromConnectionSecretKey: username
- fromConnectionSecretKey: password
- name: host
fromConnectionSecretKey: endpoint
- name: port
type: FromValue
value: "5432"
name: cloudsqlinstance
patches:
- fromFieldPath: metadata.uid
toFieldPath: spec.writeConnectionSecretToRef.name
transforms:
- string:
fmt: '%s-postgresql'
type: Format
type: string
type: FromCompositeFieldPath
- fromFieldPath: spec.parameters.storageGB
toFieldPath: spec.forProvider.settings.dataDiskSizeGb
type: FromCompositeFieldPath
EOF
The composition defined above makes sure that all CloudSQL Postgresql instances are placed in the us-central1 region. This composition fulfils the XRD previously created by creating GCP CloudSQL databases.
Warning: The authorized network CIDR 0.0.0.0/0 provided above, will allow access to the Cloud SQL from any IP and is not recommended in a production environment.
In order to make instances of a service easily discoverable and claimable by application operators, the role of the service operator creates a ClusterInstanceClass. In this particular example, the class states that claimable instances of CloudSQL Postgresql are represented by secret objects of type connection.crossplane.io/v1alpha1 with label services.apps.tanzu.vmware.com/class set to cloudsql-postgres:
kubectl apply -f -<<EOF
---
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
name: cloudsql-postgres
spec:
description:
short: GCP CloudSQL Postgresql database instances
pool:
kind: Secret
labelSelector:
matchLabels:
services.apps.tanzu.vmware.com/class: cloudsql-postgres
fieldSelector: type=connection.crossplane.io/v1alpha1
EOF
In addition, you need to grant sufficient RBAC permissions to Services Toolkit to be able to read the secrets specified by the class.
kubectl apply -f -<<EOF
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: stk-secret-reader
labels:
servicebinding.io/controller: "true"
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
EOF
Playing the role of the Service Operator, you now provision an instance of GCP CloudSQL Postgresql using the postgresqlinstances.bindable.database.example.org API managed by the XRD you previously created. Note that .spec.publishConnectionDetailsTo provides Crossplane with the name and a label for the secret that is being used to store the connection details for the newly created database. You can see that the label specified here matches the label selector defined on the ClusterInstanceClass you created in the previous step.
Run the following command:
kubectl apply -f -<<EOF
---
apiVersion: bindable.database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
name: cloudsql-postgres-db
namespace: default
spec:
parameters:
storageGB: 20
compositionSelector:
matchLabels:
provider: gcp
publishConnectionDetailsTo:
name: cloudsql-postgres-db
metadata:
labels:
services.apps.tanzu.vmware.com/class: cloudsql-postgres
EOF
Running this command will cause the creation of a CloudSQL database instance in your GCP account. You can use the gcloud CLI to verify this:
gcloud sql instances list
After the instance has been successfully created in GCP, the status of the newly created PostgreSQLInstance resource should show READY=True. This might take a few minutes. You can wait for this by running:
kubectl wait --for=condition=Ready=true postgresqlinstances.bindable.database.example.org cloudsql-postgres-db --timeout=10m
As soon as the CloudSQL Postgresql instance is ready, it is claimable by the role of the application operator as shown in the next section.
Thanks to the previously created ClusterInstanceClass, secrets representing CloudSQL Postgresql instances can now be discovered and claimed by application operators through the Tanzu CLI as shown below.
Show available classes of service instances by running:
tanzu service classes list
NAME DESCRIPTION
cloudsql-postgres GCP CloudSQL Postgresql database instances
Show claimable instances belonging to the CloudSQL Postgresql class by running:
tanzu services claimable list --class cloudsql-postgres
NAME NAMESPACE API KIND API GROUP/VERSION
cloudsql-postgres-db default Secret v1
Create a claim for the discovered instance by running:
tanzu service resource-claim create cloudsql-claim \
--resource-name cloudsql-postgres-db \
--resource-kind Secret \
--resource-api-version v1
Obtain the claim reference by running:
tanzu service resource-claim list -o wide
Expect to see the following output:
NAME READY REASON CLAIM REF
cloudsql-claim True services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:cloudsql-claim
Create an application workload that consumes the claimed CloudSQL Postgresql database by running:
Example:
tanzu apps workload create my-workload \
--git-repo https://github.com/sample-accelerators/spring-petclinic \
--git-branch main \
--git-tag tap-1.2 \
--type web \
--label app.kubernetes.io/part-of=spring-petclinic \
--annotation autoscaling.knative.dev/minScale=1 \
--env SPRING_PROFILES_ACTIVE=postgres \
--service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:cloudsql-claim
Note that --service-ref is being set to the claim reference obtained previously.
