This topic demonstrates how to use Services Toolkit to allow Tanzu Application Platform workloads to consume Azure Flexible Server for PostgreSQL. This particular topic makes use of Crossplane to manage those Flexible Server for PostgreSQL instances. As such, it can be thought of as an alternative approach to Consuming Azure Flexible Server for PostgreSQL on Tanzu Application Platform with Azure Service Operator (ASO) to achieve the similar outcomes.
Note This usecase is not currently compatible with TAP air-gapped installations.
Meet these prerequisites:
Note: In this example we use an AKS Cluster to deploy Crossplane and Tanzu Application Platform too. However, any other cluster which supports running those two systems should suffice.
Note: For the latest steps for installing Crossplane, see these instructions. For the instructions in this topic, it is important to enable support for external secret stores in Crossplane. This is currently an Alpha feature. As such, you will have to explicitly set command line flag
--enable-external-secret-storeswhen starting the Crossplane controller.
Run the following commands to install Crossplane to your existing Kubernetes cluster:
kubectl create namespace crossplane-system
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update
helm install crossplane --namespace crossplane-system crossplane-stable/crossplane \
--set 'args={--enable-external-secret-stores}'
For this topic, you do not need to install the Crossplane CLI or any additional configuration package.
To install the Azure Provider for Crossplane, run:
kubectl apply -f - <<'EOF'
apiVersion: pkg.crossplane.io/v1alpha1
kind: ControllerConfig
metadata:
name: jet-azure-config
spec:
image: crossplane/provider-jet-azure-controller:v0.12.0
args: ["-d"]
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-jet-azure
spec:
package: crossplane/provider-jet-azure:v0.12.0
controllerConfigRef:
name: jet-azure-config
EOF
After you have installed the provider, you see a new flexibleservers.dbforpostgresql.azure.jet.crossplane.io API resource available in your Kubernetes cluster. You can wait for the provider to become healthy by running:
kubectl -n crossplane-system wait provider/provider-jet-azure \
--for=condition=Healthy=True --timeout=3m
To install the Kubernetes Provider for Crossplane, run:
kubectl apply -f - <<'EOF'
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-kubernetes
spec:
package: "crossplane/provider-kubernetes:main"
EOF
This section creates a new Service Principal to be used by the Crossplane system to allow it to manage PostgreSQL Servers.
Setup some configuration in the current shell session
# Set the name of the Service Principal to be created
AZURE_SP_NAME='sql-crossplane-demo'
# Get the subscription ID
AZURE_SUBSCRIPTION_ID="$( az account show -o json | jq -r '.id' )"
Create a new Service Principal and set up the kubernetes secret
kubectl create secret generic jet-azure-creds -o yaml --dry-run=client --from-literal=creds="$(
az ad sp create-for-rbac -n "${AZURE_SP_NAME}" \
--sdk-auth \
--role "Contributor" \
--scopes "/subscriptions/${AZURE_SUBSCRIPTION_ID}" \
-o json
)" | kubectl apply -n crossplane-system -f -
Note: You’ll see the following warning
NoteWARNING: Option ‘–sdk-auth’ has been deprecated and will be removed in a future release.
which you can ignore for now. There is some context about that in this issue for the Azure CLI and this issue for the Crossplane Azure Provider.
Deploy a ProviderConfig which uses the previously created secret for the Azure crossplane provider
kubectl apply -f - <<'EOF'
apiVersion: azure.jet.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
name: default
spec:
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: jet-azure-creds
key: creds
EOF
SA=$(kubectl -n crossplane-system get sa -o name | grep provider-kubernetes | sed -e 's|serviceaccount\/|crossplane-system:|g')
kubectl create role -n crossplane-system password-manager --resource=passwords.secretgen.k14s.io --verb=create,get,update,delete
kubectl create rolebinding -n crossplane-system provider-kubernetes-password-manager --role password-manager --serviceaccount="${SA}"
kubectl apply -f - <<'EOF'
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
name: default
spec:
credentials:
source: InjectedIdentity
EOF
Now that the Azure Provider for Crossplane has been installed and configured, create a new CompositeResourceDefinition (XRD) and corresponding Composition representing individual instances of Azure PostgreSQL Server. For more information about these concepts see the Crossplane Composition documentation.
<!– Removing this note, as there are not jet XRD examples on the upbound marketplace
Note: Instead of creating your own custom XRD and Composition as shown below, you can also install an existing Crossplane configuration package for Azure that includes pre-configured XRDs and compositions for Azure PostgreSQL Servers. The primary reason for creating a new XRD and composition from scratch is to make sure the connection secrets for newly provisioned PostgreSQL Server instances support the Service Binding Specification for Kubernetes and automatic Spring Boot configuration using Spring Cloud Bindings.
–>
Create a new XRD by running:
kubectl apply -f - <<'EOF'
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xpostgresqlinstances.bindable.database.example.org
spec:
claimNames:
kind: PostgreSQLInstance
plural: postgresqlinstances
connectionSecretKeys:
- type
- provider
- host
- port
- database
- username
- password
group: bindable.database.example.org
names:
kind: XPostgreSQLInstance
plural: xpostgresqlinstances
versions:
- name: v1alpha1
referenceable: true
schema:
openAPIV3Schema:
properties:
spec:
properties:
parameters:
properties:
storageGB:
type: integer
required:
- storageGB
type: object
required:
- parameters
type: object
type: object
served: true
EOF
After the newly created XRD has been successfully reconciled, there are two new API resources available in your Kubernetes cluster, xpostgresqlinstances.bindable.database.example.org and postgresqlinstances.bindable.database.example.org.
Create a corresponding composition (not in a production environment) by running:
kubectl apply -f - <<'EOF'
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
labels:
provider: azure
name: xpostgresqlinstances.bindable.gcp.database.example.org
spec:
compositeTypeRef:
apiVersion: bindable.database.example.org/v1alpha1
kind: XPostgreSQLInstance
publishConnectionDetailsWithStoreConfigRef:
name: default
resources:
- name: dbinstance
base:
apiVersion: dbforpostgresql.azure.jet.crossplane.io/v1alpha2
kind: FlexibleServer
spec:
forProvider:
administratorLogin: myPgAdmin
administratorPasswordSecretRef:
name: ""
namespace: crossplane-system
key: password
location: westeurope
skuName: GP_Standard_D2s_v3
version: "12" #! 11,12 and 13 are supported
resourceGroupName: tap-psql-demo
writeConnectionSecretToRef:
namespace: crossplane-system
connectionDetails:
- name: type
value: postgresql
- name: provider
value: azure
- name: database
value: postgres
- name: username
fromFieldPath: spec.forProvider.administratorLogin
- name: password
fromConnectionSecretKey: "attribute.administrator_password"
- name: host
fromFieldPath: status.atProvider.fqdn
- name: port
type: FromValue
value: "5432"
patches:
- fromFieldPath: metadata.uid
toFieldPath: spec.writeConnectionSecretToRef.name
transforms:
- string:
fmt: '%s-postgresql'
type: Format
type: string
type: FromCompositeFieldPath
- type: FromCompositeFieldPath
fromFieldPath: metadata.name
toFieldPath: spec.forProvider.administratorPasswordSecretRef.name
- fromFieldPath: spec.parameters.storageGB
toFieldPath: spec.forProvider.storageMb
type: FromCompositeFieldPath
transforms:
- type: math
math:
multiply: 1024
- name: dbfwrule
base:
apiVersion: dbforpostgresql.azure.jet.crossplane.io/v1alpha2
kind: FlexibleServerFirewallRule
spec:
forProvider:
serverIdSelector:
matchControllerRef: true
#! not recommended for production deployments!
startIpAddress: 0.0.0.0
endIpAddress: 255.255.255.255
- name: password
base:
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
spec:
forProvider:
manifest:
apiVersion: secretgen.k14s.io/v1alpha1
kind: Password
metadata:
name: ""
namespace: crossplane-system
spec:
length: 64
secretTemplate:
type: Opaque
stringData:
password: $(value)
patches:
- type: FromCompositeFieldPath
fromFieldPath: metadata.name
toFieldPath: spec.forProvider.manifest.metadata.name
EOF
The composition defined above makes sure that all FlexibleServers are placed in the westeurope region and under the resource group tap-psql-demo. This composition fulfils the XRD previously created.
Warning: Setting the FlexibleServerFirewallRule to start at 0.0.0.0 and end at 255.255.255.255 will allow access to the PostgreSQL Server from any IP and is not recommended in a production environment.
In order to make instances of a service easily discoverable and claimable by Application Operators, the role of the Service Operator creates a ClusterInstanceClass. In this particular example, the class states that claimable instances of PostgreSQL instances are represented by Secret objects of type connection.crossplane.io/v1alpha1 with label services.apps.tanzu.vmware.com/class set to azure-postgres:
kubectl apply -f - <<'EOF'
apiVersion: services.apps.tanzu.vmware.com/v1alpha1
kind: ClusterInstanceClass
metadata:
name: azure-postgres
spec:
description:
short: Azure Postgresql database instances
pool:
kind: Secret
labelSelector:
matchLabels:
services.apps.tanzu.vmware.com/class: azure-postgres
fieldSelector: type=connection.crossplane.io/v1alpha1
EOF
In addition, you need to grant sufficient RBAC permissions to Services Toolkit to be able to read the secrets specified by the class.
kubectl apply -f - <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: stk-secret-reader
labels:
servicebinding.io/controller: "true"
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
EOF
Playing the role of the Service Operator, you now provision an instance of an Azure Flexible Server for PostgreSQL using the postgresqlinstances.bindable.database.example.org API managed by the XRD you previously created. Note that .spec.publishConnectionDetailsTo provides Crossplane with the name and a label for the secret that is being used to store the connection details for the newly created database. You can see that the label specified here matches the label selector defined on the ClusterInstanceClass you created in the previous step.
The PostgreSQLInstance has a dependency on a Secret where the Service Operator needs to specify the password for the admin user. Here we use Carvel’s Password API to create this Secret for us.
Run the following command:
kubectl apply -f - <<'EOF'
apiVersion: bindable.database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
name: postgresql-server
namespace: default
spec:
parameters:
#! supported storage sizes: 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768
storageGB: 32
compositionSelector:
matchLabels:
provider: azure
publishConnectionDetailsTo:
name: postgresql-server
metadata:
labels:
services.apps.tanzu.vmware.com/class: azure-postgres
EOF
Running this command will cause the creation of a Azure Flexible Server for PostgreSQL instance in your Azure account. You can use the Azure CLI to verify this:
az postgres flexible-server list -o table
After the instance has been successfully created, the status of the newly created PostgreSQLInstance resource should show READY=True. This might take a few minutes. You can wait for this by running:
kubectl wait postgresqlinstances.bindable.database.example.org/postgresql-server \
--for=condition=Ready=true --timeout=10m
As soon as the Azure Flexible Server for PostgreSQL instance is ready, it is claimable by the role of the Application Operator as shown in the next section.
Thanks to the previously created ClusterInstanceClass, Secrets representing PostgreSQL Server instances can now be discovered and claimed by Application Operators through the Tanzu CLI as shown below.
Show available classes of service instances by running:
tanzu service classes list
NAME DESCRIPTION
azure-postgres Azure Postgresql database instances
Show claimable instances belonging to the PostgreSQL Server instance class by running:
tanzu services claimable list --class azure-postgres
NAME NAMESPACE API KIND API GROUP/VERSION
postgresql-server default Secret v1
Create a claim for the discovered instance by running:
tanzu service resource-claim create postgresql-server-claim \
--resource-name postgresql-server\
--resource-kind Secret \
--resource-api-version v1
Obtain the claim reference by running:
tanzu service resource-claim list -o wide
Expect to see the following output:
NAME READY REASON CLAIM REF
postgresql-server-claim True services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:postgresql-server-claim
Create an application workload that consumes the claimed PostgreSQL Server instance by running:
Example:
tanzu apps workload create my-workload \
--git-repo https://github.com/sample-accelerators/spring-petclinic \
--git-branch main \
--git-tag tap-1.2 \
--type web \
--label app.kubernetes.io/part-of=spring-petclinic \
--annotation autoscaling.knative.dev/minScale=1 \
--env SPRING_PROFILES_ACTIVE=postgres \
--service-ref db=services.apps.tanzu.vmware.com/v1alpha1:ResourceClaim:postgresql-server-claim
Note that --service-ref is being set to the claim reference obtained previously.
