Consuming GCP CloudSQL on Tanzu Application Platform with Crossplane

Introduction

This topic demonstrates how to use Services Toolkit to allow Tanzu Application Platform workloads to consume GCP CloudSQL PostgreSQL databases. This particular guide makes use of Crossplane to manage CloudSQL instances in GCP. As such, it can be thought of as an alternative approach to Consuming Google Cloud SQL on Tanzu Application Platform (TAP) with Config Connector to achieve the same outcomes.

Note This usecase is not currently compatible with TAP air-gapped installations.

Prerequisites

Meet these prerequisites:

Install Crossplane

Note

: For the latest steps for installing Crossplane, see these instructions. For the instructions in this topic, it is important to enable support for external secret stores in Crossplane. This is currently an Alpha feature. As such, you will have to explicitly set command line flag --enable-external-secret-stores when starting the Crossplane controller.

Run the following commands to install Crossplane to your existing Kubernetes cluster:

kubectl create namespace crossplane-system

helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

helm install crossplane --namespace crossplane-system crossplane-stable/crossplane \
  --set 'args={--enable-external-secret-stores}'

For this topic, you do not need to install the Crossplane CLI or any additional configuration package.

Install GCP Provider for Crossplane

To install the GCP Provider for Crossplane, run:

kubectl apply -f -<<EOF
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: crossplane-provider-gcp
spec:
  package: crossplane/provider-gcp:v0.21.0
EOF

After you have installed the provider, you see a new cloudsqlinstances.database.gcp.crossplane.io API resource available in your Kubernetes cluster. See the health of the installed provider by running:

kubectl get provider.pkg.crossplane.io crossplane-provider-gcp

Configure GCP Provider

This section creates a new GCP Service Account and gives it permissions to manage CloudSQL databases which are necessary to use Crossplane to manage CloudSQL instances.

  1. Create a new GCP ServiceAccount, give it Cloud SQL Admin and create a key file:

    PROJECT_ID=<GCP Project ID>
    SA_NAME=crossplane-cloudsql
    
    gcloud iam service-accounts create "${SA_NAME}" --project "${PROJECT_ID}"
    gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
      --role="roles/cloudsql.admin" \
      --member "serviceAccount:${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
    gcloud iam service-accounts keys create creds.json --project "${PROJECT_ID}" --iam-account "${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
    
  2. Create a new secret from the key file by running:

    kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./creds.json
    
  3. Delete the key file by running:

    rm -f creds.json
    
  4. Configure the GCP provider to use the newly created secret by running:

    kubectl apply -f -<<EOF
    apiVersion: gcp.crossplane.io/v1beta1
    kind: ProviderConfig
    metadata:
      name: default
    spec:
      projectID: ${PROJECT_ID}
      credentials:
        source: Secret
        secretRef:
          namespace: crossplane-system
          name: gcp-creds
          key: creds
    EOF
    

Define Composite Resource Types

Now that the GCP provider for Crossplane has been installed and configured, create a new CompositeResourceDefinition (XRD) and corresponding Composition representing individual instances of CloudSQL Postgresql. For more information about these concepts see the Crossplane Composition documentation.

Note: Instead of creating your own custom XRD and Composition as shown below, you can also install an existing Crossplane configuration package for GCP that includes pre-configured XRDs and compositions for CloudSQL. The primary reason for creating a new XRD and composition from scratch is to make sure the connection secrets for newly provisioned CloudSQL Postgresql instances support the Service Binding Specification for Kubernetes and automatic Spring Boot configuration using Spring Cloud Bindings.

  1. Create a new XRD by running:

    kubectl apply -f -<<EOF
    ---
    apiVersion: apiextensions.crossplane.io/v1
    kind: CompositeResourceDefinition
    metadata:
      name: xpostgresqlinstances.bindable.database.example.org
    spec:
      claimNames:
        kind: PostgreSQLInstance
        plural: postgresqlinstances
      connectionSecretKeys:
      - type
      - provider
      - host
      - port
      - database
      - username
      - password
      group: bindable.database.example.org
      names:
        kind: XPostgreSQLInstance
        plural: xpostgresqlinstances
      versions:
      - name: v1alpha1
        referenceable: true
        schema:
          openAPIV3Schema:
            properties:
              spec:
                properties:
                  parameters:
                    properties:
                      storageGB:
                        type: integer
                    required:
                    - storageGB
                    type: object
                required:
                - parameters
                type: object
            type: object
        served: true
    EOF
    

    After the newly created XRD has been successfully reconciled, there are two new API resources available in your Kubernetes cluster, xpostgresqlinstances.bindable.database.example.org and postgresqlinstances.bindable.database.example.org. The XRD created is agnostic to the underlying cloud managed service, so could also be fulfilled by a Composition that makes use of AWS RDS Postgresql or Azure Database for PostgreSQL.

  2. Create a corresponding composition (not in a production environment) by running:

    kubectl apply -f -<<EOF
    ---
    apiVersion: apiextensions.crossplane.io/v1
    kind: Composition
    metadata:
      labels:
        provider: gcp
      name: xpostgresqlinstances.bindable.gcp.database.example.org
    spec:
      compositeTypeRef:
        apiVersion: bindable.database.example.org/v1alpha1
        kind: XPostgreSQLInstance
      publishConnectionDetailsWithStoreConfigRef:
        name: default
      resources:
      - base:
          apiVersion: database.gcp.crossplane.io/v1beta1
          kind: CloudSQLInstance
          spec:
            forProvider:
              databaseVersion: POSTGRES_14
              region: us-central1
              settings:
                dataDiskType: PD_SSD
                ipConfiguration:
                  authorizedNetworks:
                  - value: 0.0.0.0/0 # not recommended for production deployments!
                  ipv4Enabled: true
                tier: db-custom-1-3840
            writeConnectionSecretToRef:
              namespace: crossplane-system
        connectionDetails:
        - name: type
          value: postgresql
        - name: provider
          value: gcp
        - name: database
          value: postgres
        - fromConnectionSecretKey: username
        - fromConnectionSecretKey: password
        - name: host
          fromConnectionSecretKey: endpoint
        - name: port
          type: FromValue
          value: "5432"
        name: cloudsqlinstance
        patches:
        - fromFieldPath: metadata.uid
          toFieldPath: spec.writeConnectionSecretToRef.name
          transforms:
          - string:
              fmt: '%s-postgresql'
              type: Format
            type: string
          type: FromCompositeFieldPath
        - fromFieldPath: spec.parameters.storageGB
          toFieldPath: spec.forProvider.settings.dataDiskSizeGb
          type: FromCompositeFieldPath
    EOF
    

    The composition defined above makes sure that all CloudSQL Postgresql instances are placed in the us-central1 region. This composition fulfils the XRD previously created by creating GCP CloudSQL databases.

    Warning: The authorized network CIDR 0.0.0.0/0 provided above, will allow access to the Cloud SQL from any IP and is not recommended in a production environment.

Create an Instance Class

In order to make instances of a service easily discoverable and claimable by application operators, the role of the service operator creates a ClusterInstanceClass. In this particular example, the class states that claimable instances of CloudSQL Postgresql are represented by secret objects of type connection.crossplane.io/v1alpha1 with label services.apps.tanzu.vmware.com/class set to cloudsql-postgres:

kubectl apply -f -<<EOF
---
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
  name: cloudsql-postgres
spec:
  description:
    short: GCP CloudSQL Postgresql database instances
  pool:
    kind: Secret
    labelSelector:
      matchLabels:
        services.apps.tanzu.vmware.com/class: cloudsql-postgres
    fieldSelector: type=connection.crossplane.io/v1alpha1
EOF

In addition, you need to grant sufficient RBAC permissions to Services Toolkit to be able to read the secrets specified by the class.

kubectl apply -f -<<EOF
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: stk-secret-reader
  labels:
    servicebinding.io/controller: "true"
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
EOF

Provision GCP CloudSQL Postgresql Instance

Playing the role of the Service Operator, you now provision an instance of GCP CloudSQL Postgresql using the postgresqlinstances.bindable.database.example.org API managed by the XRD you previously created. Note that .spec.publishConnectionDetailsTo provides Crossplane with the name and a label for the secret that is being used to store the connection details for the newly created database. You can see that the label specified here matches the label selector defined on the ClusterInstanceClass you created in the previous step.

Run the following command:

kubectl apply -f -<<EOF
---
apiVersion: bindable.database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
  name: cloudsql-postgres-db
  namespace: default
spec:
  parameters:
    storageGB: 20
  compositionSelector:
    matchLabels:
      provider: gcp
  publishConnectionDetailsTo:
    name: cloudsql-postgres-db
    metadata:
      labels:
        services.apps.tanzu.vmware.com/class: cloudsql-postgres
EOF

Running this command will cause the creation of a CloudSQL database instance in your GCP account. You can use the gcloud CLI to verify this:

gcloud sql instances list

After the instance has been successfully created in GCP, the status of the newly created PostgreSQLInstance resource should show READY=True. This might take a few minutes. You can wait for this by running:

kubectl wait --for=condition=Ready=true postgresqlinstances.bindable.database.example.org cloudsql-postgres-db --timeout=10m

As soon as the CloudSQL Postgresql instance is ready, it is claimable by the role of the application operator as shown in the next section.

Claim the CloudSQL Postgresql instance and connect to it from the Tanzu Application Platform Workload

Thanks to the previously created ClusterInstanceClass, secrets representing CloudSQL Postgresql instances can now be discovered and claimed by application operators through the Tanzu CLI as shown below.

  1. Show available classes of service instances by running:

    tanzu service classes list
    
      NAME                 DESCRIPTION
      cloudsql-postgres    GCP CloudSQL Postgresql database instances
    
  2. Show claimable instances belonging to the CloudSQL Postgresql class by running:

    tanzu services claimable list --class cloudsql-postgres
    
      NAME                    NAMESPACE  API KIND  API GROUP/VERSION
      cloudsql-postgres-db    default    Secret    v1
    
  3. Create a claim for the discovered instance by running:

    tanzu service claim create cloudsql-claim \
      --resource-name cloudsql-postgres-db \
      --resource-kind Secret \
      --resource-api-version v1
    
  4. Obtain the claim reference by running:

    tanzu service claim list -o wide
    

    Expect to see the following output:

    NAME                     READY  REASON  CLAIM REF
    cloudsql-claim           True           services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:cloudsql-claim
    
  5. Create an application workload that consumes the claimed CloudSQL Postgresql database by running:

    Example:

    tanzu apps workload create my-workload \
      --git-repo https://github.com/sample-accelerators/spring-petclinic \
      --git-branch main \
      --git-tag tap-1.2 \
      --type web \
      --label app.kubernetes.io/part-of=spring-petclinic \
      --annotation autoscaling.knative.dev/minScale=1 \
      --env SPRING_PROFILES_ACTIVE=postgres \
      --service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:cloudsql-claim
    

    Note that --service-ref is being set to the claim reference obtained previously.

check-circle-line exclamation-circle-line close-line
Scroll to top icon