Create conventions with Cartographer Conventions

This topic describes how you can create and deploy custom conventions to the Tanzu Application Platform by using Cartographer Conventions.

Introduction

Tanzu Application Platform helps developers transform their code into containerized workloads with a URL. The Supply Chain Choreographer for Tanzu manages this transformation. For more information, see Supply Chain Choreographer.

Cartographer Conventions is a key component of the supply chain compositions the choreographer calls into action. Cartographer Conventions enables people in operational roles to efficiently apply their expertise. They can specify the runtime best practices, policies, and conventions of their organization to workloads as they are created on the platform.

You can see the power of this component when the conventions of an organization are applied consistently, at scale, without slowing down application developers.

Opinions and policies vary from organization to organization. Cartographer Convention supports the creation of custom conventions to meet the unique operational needs and requirements of an organization.

Before jumping into the details of creating a custom convention, you can view two distinct components of Cartographer Conventions:

Convention server

The convention server is the component that applies a convention already defined on the server. For a Go example of creating a convention server to add Spring Boot conventions, see spring-convention-server in GitHub. The resource that structures the request body of the request and response from the server is the PodConventionContext.

The PodConventionContext is a webhooks.conventions.carto.run/v1alpha1 type that defines the structure used to communicate internally by the webhook convention server. PodConventionContext does not exist on the Kubernetes API Server.

PodConventionContext is a wrapper for two types:

  • PodConventionContextSpec which acts as a wrapper for a PodTemplateSpec and a list of ImageConfigs provided in the request body of the server.
  • PodConventionContextStatus which is a status type used to represent the current status of the context retrieved by the request.

For information about an example PodConventionContext, see PodConventionContext in GitHub. For information about a Convention server and the structure of these types, see OpenAPI Spec in GitHub.

How the convention server works

Each convention server can host one or more conventions. The application of each convention by a convention server are controlled conditionally. The conditional criteria governing the application of a convention are customizable and are based on the evaluation of a custom Kubernetes resource called PodIntent. PodIntent is the vehicle through which Cartographer Conventions as a whole delivers its value.

A PodIntent is created, or updated if already existing, when a workload is run by using a Tanzu Application Platform supply chain. The custom resource includes both the PodTemplateSpec and the Open Containers Initiative (OCI) image metadata associated with a workload. See the Kubernetes documentation. The conditional criteria for a convention are based on any property or value found in the PodTemplateSpec or the OCI image metadata available in the PodIntent.

If a convention’s criteria are met, the convention server enriches the PodTemplateSpec in the PodIntent. The convention server also updates the status section of the PodIntent with the name of the convention that is applied. Afterward you can learn which conventions were applied to the workload.

To provide flexibility in how conventions are organized, you can deploy multiple convention servers. Each server can contain a convention or set of conventions focused on a specific class of runtime modifications, on a specific language framework, and so on. How the conventions are organized, grouped, and deployed is up to you and the needs of your organization.

Convention servers deployed to the cluster do not take action unless the second component of Cartographer Conventions, the convention controller, triggers them to take action.

The convention controller

The convention controller is the orchestrator of one or many convention servers deployed to the cluster. There are resources available on the conventions.carto.run/v1aplha1 API that allow the controller to perform its functions. These resources include:

  • ClusterPodConvention, which is a resource type that allows the conventions author to register a webhook server with the controller by using its spec.webhook field.

    ...
    spec:
      selectorTarget: PodTemplateSpec # optional field with options, defaults to PodTemplateSpec
      selectors: # optional, defaults to match all workloads
      - <metav1.LabelSelector>
      webhook:
        certificate:
          name: sample-cert
          namespace: sample-conventions
        clientConfig:
          <admissionregistrationv1.WebhookClientConfig>
    
  • PodIntent, which is a conventions.carto.run/v1alpha1 resource type. PodIntent continuously reconciles and applies decorations to a workload PodTemplateSpec, exposing the enriched PodTemplateSpec on its status. Updating the status of the PodIntent does not cause side effects on the cluster.

    As key types defined on the conventions.carto.run API, the ClusterPodConvention and PodIntent resources are both present on the Kubernetes API Server. clusterpodconventions.conventions.carto.run is used to query ClusterPodConvention. podintents.conventions.carto.run is used to query PodIntent.

How the convention services’s controller works

When the Supply Chain Choreographer creates or updates a PodIntent for a workload, the convention controller retrieves the OCI image metadata from the repository containing the workload’s images and sets it in the PodIntent.

The convention controller then uses a webhook architecture to pass the PodIntent to each convention server deployed to the cluster. The controller orchestrates the processing of the PodIntent by the convention servers sequentially, based on the priority value that is set on the convention server. For more information, see ClusterPodConvention.

After all convention servers have finished processing a PodIntent for a workload, the convention controller updates the PodIntent with the latest version of the PodTemplateSpec and sets PodIntent.status.conditions[].status=True where PodIntent.status.conditions[].type=Ready.

This status change signals the Supply Chain Choreographer that Cartographer Conventions is finished with its work. The status change also executes whatever steps are waiting in the supply chain.

Getting started

With this high-level understanding of Cartographer Conventions components, you are ready to create and deploy a custom convention.

Note

This topic covers developing conventions using Go, but you can use other languages that adhere to the specifications.

Prerequisites

The following prerequisites must be met before a convention is developed and deployed:

  • The Kubernetes CLI tool (kubectl) CLI is installed. For more information, see the Kubernetes documentation.
  • Tanzu Application Platform prerequisites are installed. For more information, see Prerequisites
  • Tanzu Application Platform components are installed. For more information, see the Installing the Tanzu CLI.
  • The default supply chain is installed. Download Supply Chain Security Tools for VMware Tanzu from the Broadcom Support Portal.
  • Your kubeconfig context is set to the Tanzu Application Platform-enabled cluster:

    kubectl config use-context CONTEXT_NAME
    
  • The ko CLI is installed. Use GitHub to install the ko CLI. See the google/ko GitHub repository. These instructions use ko to build an image. If there is an existing image or build process, ko is optional.

Define convention criteria

The server.go file contains the configuration for the server and the logic the server applies when a workload matches the defined criteria. For example, adding a Prometheus sidecar to web applications, or adding a workload-type=spring-boot label to any workload that has metadata, indicating it is a Spring Boot app.

Important

For this example, the package model defines resource types.

  1. The example server.go configures the ConventionHandler to ingest the webhook requests from the convention controller. See PodConventionContext. Here the handler must only deal with the existing PodTemplateSpec and ImageConfig.

    ...
    import (
      corev1 "k8s.io/api/core/v1"
    )
    ...
    func ConventionHandler(template *corev1.PodTemplateSpec, images []model.ImageConfig) ([]string, error) {
       // Create custom conventions
    }
    ...
    

    Where:

    • template is the predefined PodTemplateSpec that the convention edits. For more information about PodTemplateSpec, see the Kubernetes documentation.
    • images are the ImageConfig used as reference to make decisions in the conventions. In this example, the type was created within the model package.
  2. The example server.go also configures the convention server to listen for requests:

    ...
    import (
       "context"
       "fmt"
       "log"
       "net/http"
       "os"
       ...
    )
    ...
    func main() {
       ctx := context.Background()
       port := os.Getenv("PORT")
       if port == "" {
           port = "9000"
       }
       http.HandleFunc("/", webhook.ServerHandler(convention.ConventionHandler))
       log.Fatal(webhook.NewConventionServer(ctx, fmt.Sprintf(":%s", port)))
    }
    ...
    

    Where:

    • PORT is a possible environment variable, for this example, defined in the deployment.
    • ServerHandler is the handler function called when any request comes to the server.
    • NewConventionServer is the function in charge of configuring and creating the HTTP webhook server.
    • port is the calculated port of the server to listen for requests. It must match the deployment if the PORT variable is not defined in it.
    • The path or pattern (default to /) is the convention server’s default path. If it is changed, it must be changed in the ClusterPodConvention.
    Note

    The Server Handler, func ConventionHandler(...), and the configure or start web server, func NewConventionServer(...), is defined in the convention controller in the webhook package, but you can use a custom one.

  3. Creating the Server Handler, which handles the request from the convention controller with the PodConventionContext serialized to JSON.

    package webhook
    ...
    func ServerHandler(conventionHandler func(template *corev1.PodTemplateSpec, images []model.ImageConfig) ([]string, error)) http.HandlerFunc {
        return func(w http.ResponseWriter, r *http.Request) {
            ...
            // Check request method
            ...
            // Decode the PodConventionContext
            podConventionContext := &model.PodConventionContext{}
            err = json.Unmarshal(body, &podConventionContext)
            if err != nil {
                w.WriteHeader(http.StatusBadRequest)
                return
            }
            // Validate the PodTemplateSpec and ImageConfig
            ...
            // Apply the conventions
            pts := podConventionContext.Spec.Template.DeepCopy()
            appliedConventions, err := conventionHandler(pts, podConventionContext.Spec.Images)
            if err != nil {
                w.WriteHeader(http.StatusInternalServerError)
                return
            }
            // Update the applied conventions and status with the new PodTemplateSpec
            podConventionContext.Status.AppliedConventions = appliedConventions
            podConventionContext.Status.Template = *pts
            // Return the updated PodConventionContext
            w.Header().Set("Content-Type", "application/json")
            w.WriteHeader(http.StatusOK)
            json.NewEncoder(w).Encode(podConventionContext)
        }
    }
    ...
    
  4. Configure and start the web server by defining the NewConventionServer function, which starts the server with the defined port and current context. The server uses the .crt and .key files to handle TLS traffic.

    package webhook
    ...
    // Watch handles the security by certificates.
    type certWatcher struct {
        CrtFile string
        KeyFile string
    
        m       sync.Mutex
        keyPair *tls.Certificate
    }
    func (w *certWatcher) Load() error {
        // Creates a X509KeyPair from PEM encoded client certificate and private key.
        ...
    }
    func (w *certWatcher) GetCertificate() *tls.Certificate {
        w.m.Lock()
        defer w.m.Unlock()
    
        return w.keyPair
    }
    ...
    func NewConventionServer(ctx context.Context, addr string) error {
        // Define a health check endpoint to readiness and liveness probes.
        http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
            w.WriteHeader(http.StatusOK)
        })
    
        if err := watcher.Load(); err != nil {
            return err
        }
        // Defines the server with the TLS configuration.
        server := &http.Server{
            Addr: addr,
            TLSConfig: &tls.Config{
                GetCertificate: func(_ *tls.ClientHelloInfo) (*tls.Certificate, error) {
                    cert := watcher.GetCertificate()
                    return cert, nil
                },
                PreferServerCipherSuites: true,
                MinVersion:               tls.VersionTLS13,
            },
            BaseContext: func(_ net.Listener) context.Context {
                return ctx
            },
        }
        go func() {
            <-ctx.Done()
            server.Close()
        }()
    
        return server.ListenAndServeTLS("", "")
    }
    

Define the convention behavior

Any property or value within the PodTemplateSpec or OCI image metadata associated with a workload defines the criteria for applying conventions. See PodTemplateSpec in the Kubernetes documentation. The following are a few examples.

Matching criteria by labels or annotations

The conventions.carto.run/v1alpha1 API allows convention authors to use the selectorTarget field which complements the ClusterPodConvention matchers to specify whether to consider labels on either one of the following available options:

  • PodTemplateSpec

      ...
      template:
        metadata:
          labels:
            awesome-label: awesome-value
          annotations:
            awesome-annotation: awesome-value
      ...
    
  • PodIntent

        ...
        kind: PodIntent
        metadata:
          name: test-pod
          labels:
            environment: production
            ...
    

The selectorTarget field is configured on the ClusterPodConvention as follows:

...
spec:
  selectorTarget: PodIntent # optional, defaults to PodTemplateSpec
  selectors: # optional, defaults to match all workloads
  - <metav1.LabelSelector>
  webhook:
    certificate:
      name: sample-cert
      namespace: sample-conventions
    clientConfig:
      <admissionregistrationv1.WebhookClientConfig>

If you do not provide a value for this optional field while using the conventions.carto.run/v1alpha1 API, the default value is set to PodTemplateSpec.

Matching criteria by environment variables

When using environment variables to define whether the convention is applicable, it must be present in the PodTemplateSpec, spec, containers, and env to validate the value.

  • PodTemplateSpec

    ...
    template:
      spec:
        containers:
          - name: awesome-container
            env:
    ...
    
  • Handler

    package convention
    ...
    func conventionHandler(template *corev1.PodTemplateSpec, images []model.ImageConfig) ([]string, error) {
        if len(template.Spec.Containers[0].Env) == 0 {
            template.Spec.Containers[0].Env = append(template.Spec.Containers[0].Env, corev1.EnvVar{
                Name: "MY_AWESOME_VAR",
                Value: "MY_AWESOME_VALUE",
            })
            return []string{"awesome-envs-convention"}, nil
        }
        return []string{}, nil
        ...
    }
    

Matching criteria by image metadata

For each image contained within the PodTemplateSpec, the convention controller fetches the OCI image metadata and known bill of materials (BOMs), providing it to the convention server as ImageConfig. This metadata is introspected to make decisions about how to configure the PodTemplateSpec.

Configure and install the convention server

The server.yaml defines the Kubernetes components that enable the convention server in the cluster. The next definitions are within the file.

  1. A namespace is created for the convention server components and has the required objects to run the server. It is used in the ClusterPodConvention section to indicate to the controller where the server is.

    ...
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: awesome-convention
    ---
    ...
    
  2. (Optional) A certificate manager Issuer is created to issue the> certificate needed for TLS communication.

    ...
    ---
    # The following manifests contain a self-signed issuer CR and a certificate CR.
    # More document can be found at https://docs.cert-manager.io
    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      name: awesome-selfsigned-issuer
      namespace: awesome-convention
    spec:
      selfSigned: {}
    ---
    ...
    
  3. (Optional) A self-signed Certificate is created.

    ...
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      name: awesome-webhook-cert
      namespace: awesome-convention
    spec:
      subject:
        organizations:
        - vmware
        organizationalUnits:
        - tanzu
      commonName: awesome-webhook.awesome-convention.svc
      dnsNames:
      - awesome-webhook.awesome-convention.svc
      - awesome-webhook.awesome-convention.svc.cluster.local
      issuerRef:
        kind: Issuer
        name: awesome-selfsigned-issuer
      secretName: awesome-webhook-cert
      revisionHistoryLimit: 10
    ---
    ...
    
  4. A Kubernetes Deployment is created to run the webhook from. The Service uses the container port defined by the Deployment to expose the server.

    ...
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: awesome-webhook
      namespace: awesome-convention
    spec:
      replicas: 1
      selector:
        matchLabels:
        app: awesome-webhook
      template:
        metadata:
          labels:
            app: awesome-webhook
        spec:
          containers:
          - name: webhook
            # Set the prebuilt image of the convention or use ko to build an image from code.
            # see https://github.com/google/ko
            image: ko://awesome-repo/awesome-user/awesome-convention
          env:
          - name: PORT
            value: "8443"
          ports:
          - containerPort: 8443
            name: webhook
          livenessProbe:
            httpGet:
              scheme: HTTPS
              port: webhook
              path: /healthz
          readinessProbe:
            httpGet:
              scheme: HTTPS
              port: webhook
              path: /healthz
          volumeMounts:
          - name: certs
            mountPath: /config/certs
            readOnly: true
        volumes:
        - name: certs
          secret:
            defaultMode: 420
            secretName: awesome-webhook-cert
    ---
    ...
    
  5. A Kubernetes Service to expose the convention deployment is created. For this example, the exposed port is the default 443. If you change the port, the ClusterPodConvention must be updated.

    ...
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: awesome-webhook
      namespace: awesome-convention
      labels:
        app: awesome-webhook
    spec:
      selector:
        app: awesome-webhook
      ports:
        - protocol: TCP
          port: 443
          targetPort: webhook
    ---
    ...
    
  6. The ClusterPodConvention adds the convention to the cluster to make it available for the convention controller:

    ...
    ---
    apiVersion: conventions.carto.run/v1alpha1
    kind: ClusterPodConvention
    metadata:
      name: awesome-convention
      annotations:
        conventions.carto.run/inject-ca-from: "awesome-convention/awesome-webhook-cert"
    spec:
      webhook:
        clientConfig:
          service:
            name: awesome-webhook
            namespace: awesome-convention
            # path: "/" # default
            # port: 443 # default
    
    Important

    The annotations block is only needed if you use a self-signed certificate. See the cert-manager documentation.

Deploy a convention server

To deploy a convention server:

  1. Build and install the convention:

    ko tool
    To build and deploy the convention, use the ko tool on GitHub. Use the ko tool to compile your Go code into a Docker image and push it to the registry KO_DOCKER_REGISTRY.
    ko apply -f dist/server.yaml
    
    Other tool
    If a different tool builds the image, the configuration is also applied by using either kubectl or kapp, setting the correct image in the deployment descriptor.

    For kubectl, run:

    kubectl apply -f server.yaml
    

    For kapp, run:

    kapp deploy -y -a awesome-convention -f server.yaml
    
  2. If the server is running, kubectl get all -n awesome-convention returns output such as:

    NAME                                       READY   STATUS    RESTARTS   AGE
    pod/awesome-webhook-1234567890-12345       1/1     Running   0          8h
    
    NAME                          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
    service/awesome-webhook       ClusterIP   10.56.12.49   <none>        443/TCP   28h
    
    NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/awesome-webhook       1/1     1            1           28h
    
    NAME                                             DESIRED   CURRENT   READY   AGE
    replicaset.apps/awesome-webhook-1234563213       0         0         0       23h
    replicaset.apps/awesome-webhook-5b79d5cb59       0         0         0       28h
    replicaset.apps/awesome-webhook-5bf557c9f8       1         1         1       20h
    replicaset.apps/awesome-webhook-77c647c987       0         0         0       23h
    replicaset.apps/awesome-webhook-79d9c6f74c       0         0         0       23h
    replicaset.apps/awesome-webhook-7d9d667b8d       0         0         0       9h
    replicaset.apps/awesome-webhook-8668664d75       0         0         0       23h
    replicaset.apps/awesome-webhook-9b6957476        0         0         0       24h
    
  3. To verify that the conventions are applied, ensure that the PodIntent of a workload that matches the convention criteria:

    kubectl -o yaml get podintents.conventions.apps.tanzu.vmware.co awesome-app
    

    Example YAML:

    apiVersion: conventions.carto.run/v1alpha1
    kind: PodIntent
    metadata:
      creationTimestamp: "2021-10-07T13:30:00Z"
      generation: 1
      labels:
        app.kubernetes.io/component: intent
        carto.run/cluster-supply-chain-name: awesome-supply-chain
        carto.run/cluster-template-name: convention-template
        carto.run/component-name: config-provider
        carto.run/template-kind: ClusterConfigTemplate
        carto.run/workload-name: awesome-app
        carto.run/workload-namespace: default
      name: awesome-app
      namespace: default
    ownerReferences:
    - apiVersion: carto.run/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: Workload
      name: awesome-app
      uid: "********"
    resourceVersion: "********"
    uid: "********"
    spec:
    imagePullSecrets:
      - name: registry-credentials
        serviceAccountName: default
        template:
          metadata:
            annotations:
              developer.conventions/target-containers: workload
            labels:
              app.kubernetes.io/component: run
              app.kubernetes.io/part-of: awesome-app
              carto.run/workload-name: awesome-app
          spec:
            containers:
            - image: awesome-repo.com/awesome-project/awesome-app@sha256:********
              name: workload
              resources: {}
              securityContext:
              runAsUser: 1000
    status:
      conditions:
      - lastTransitionTime: "2021-10-07T13:30:00Z"
        status: "True"
        type: ConventionsApplied
      - lastTransitionTime: "2021-10-07T13:30:00Z"
        status: "True"
        type: Ready
    observedGeneration: 1
    template:
      metadata:
        annotations:
          awesome-annotation: awesome-value
          conventions.carto.run/applied-conventions: |-
            awesome-label-convention
            awesome-annotation-convention
            awesome-envs-convention
            awesome-image-convention
            developer.conventions/target-containers: workload
        labels:
          awesome-label: awesome-value
          app.kubernetes.io/component: run
          app.kubernetes.io/part-of: awesome-app
          carto.run/workload-name: awesome-app
          conventions.carto.run/framework: go
      spec:
        containers:
        - env:
          - name: MY_AWESOME_VAR
            value: "MY_AWESOME_VALUE"
          image: awesome-repo.com/awesome-project/awesome-app@sha256:********
          name: workload
          ports:
            - containerPort: 8080
              protocol: TCP
          resources: {}
          securityContext:
            runAsUser: 1000
    

Next Steps

Consider using different matching criteria for the conventions or enhance the supply chain with multiple conventions.

check-circle-line exclamation-circle-line close-line
Scroll to top icon