Out of the Box Supply Chain with Testing for Supply Chain Choreographer

This topic provides an overview of Out of the Box Supply Chain with Testing for Supply Chain Choreographer.

This package contains Cartographer Supply Chains that tie together a series of Kubernetes resources that drive a developer-provided workload from source code to a Kubernetes configuration ready to be deployed to a cluster. It passes the source code forward to image building only if the testing pipeline supplied by the developers runs successfully.

This package includes all the capabilities of the Out of the Box Supply Chain Basic, but adds testing with Tekton.

For workloads that use either source code or prebuilt images, it performs the following:

Building from source code:
1. Watching a Git Repository or local directory for changes
2. Running tests from a developer-provided Tekton pipeline
3. Building a container image out of the source code with Buildpacks
4. Applying operator-defined conventions to the container definition
5. Deploying the application to the same cluster
6. (Beta) Alternatively, outputting a Carvel Package containing the application to a Git Repository. See Carvel Package Supply Chains.
Using a prebuilt application image:
1. Applying operator-defined conventions to the container definition
2. Creating a deliverable object for deploying the application to a cluster
3. (Beta) Alternatively, outputting a Carvel Package containing the application to a Git Repository. See Carvel Package Supply Chains.

Prerequisites

To use this supply chain, ensure:

Out of the Box Templates is installed.
Out of the Box Supply Chain With Testing is installed.
Out of the Box Supply Chain With Testing and Scanning is NOT installed.
Developer namespace is configured with the objects per Out of the Box Supply Chain Basic guidance. This supply chain is in addition to the basic one.
(optionally) Install Out of the Box Delivery Basic, if you are willing to deploy the application to the same cluster as the workload and supply chains.

To verify that you have the right set of supply chains installed (that is, the one with Scanning and not the one with testing), run:

tanzu apps cluster-supply-chain list

NAME                      LABEL SELECTOR
source-test-to-url        apps.tanzu.vmware.com/has-tests=true,apps.tanzu.vmware.com/workload-type=web
source-to-url             apps.tanzu.vmware.com/workload-type=web

If you see source-test-scan-to-url in the list, the setup is wrong: you must not have the source-test-scan-to-url installed at the same time as source-test-to-url.

Developer namespace

As mentioned in the prerequisites section, this supply chain builds on the previous Out of the Box Supply Chain, so only additions are included here.

To ensure that you have configured the namespace correctly, the namespace must have the following objects in it (including the ones marked with ‘new’ whose explanation and details are provided below):

registries secrets: Kubernetes secrets of type kubernetes.io/dockerconfigjson that contain credentials for pushing and pulling the container images built by the supply chain and the installation of Tanzu Application Platform.

For more information, see Out of the Box Supply Chain Basic.
service account: The identity to be used for any interaction with the Kubernetes API made by the supply chain

For more information, see Out of the Box Supply Chain Basic.
rolebinding: Grant to the identity the necessary roles for creating the resources prescribed by the supply chain.

For more information, see Out of the Box Supply Chain Basic.
Tekton pipeline (new): A pipeline runs whenever the supply chain hits the stage of testing the source code.

The following sections provide details about the new objects compared to Out of the Box Supply Chain Basic.

Updates to the developer Namespace

For source code testing to be present in the supply chain, a Tekton Pipeline must exist in the same namespace as the Workload so that, at the right moment, the Tekton PipelineRun object that gets created to run the tests can reference such developer-provided Pipeline.

Aside from the objects previously defined in the Out of the Box Supply Chain Basic section, you must include one more:

tekton/Pipeline: the definition of a series of tasks to run against the source code that was found by earlier resources in the Supply Chain.

Tekton/Pipeline

By default, the workload is matched to the corresponding pipeline to run using labels. Pipelines must have the label apps.tanzu.vmware.com/pipeline: test at a minimum, but you can add additional labels for granularity. This provides a default match if no other labels are provided. The pipeline expects two parameters:

source-url, an HTTP address where a .tar.gz file containing all the source code to be tested is found
source-revision, the revision of the commit or image reference (in case of workload.spec.source.image being set instead of workload.spec.source.git)

For example:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: developer-defined-tekton-pipeline
  labels:
    apps.tanzu.vmware.com/pipeline: test      # (!) required
spec:
  params:
    - name: source-url                        # (!) required
    - name: source-revision                   # (!) required
  tasks:
    - name: test
      params:
        - name: source-url
          value: $(params.source-url)
        - name: source-revision
          value: $(params.source-revision)
      taskSpec:
        params:
          - name: source-url
          - name: source-revision
        # Remove this step template for Tanzu Application Platform v1.8.0 when running on OpenShift.
        stepTemplate:
          securityContext:
            allowPrivilegeEscalation: false
            runAsUser: 1000
            capabilities:
              drop:
                - ALL
            seccompProfile:
              type: "RuntimeDefault"
            runAsNonRoot: true
        steps:
          - name: test
            image: gradle
            script: |-
              cd `mktemp -d`
              wget -qO- $(params.source-url) | tar xvz -m
              ./mvnw test

Currently, changes to the developer-provided Tekton Pipeline do not automatically trigger a re-run of the pipeline. That is, a new Tekton PipelineRun is not automatically created if a field in the Pipeline object is changed. As a workaround, delete the latest PipelineRun to trigger a re-run.

Note
If your cluster has Pod Security Admission enabled, you must update all pipeline tasks to adhere to the admission policy. For more information, see Tekton Tasks on a cluster with Pod Security Admission.

Allow multiple Tekton pipelines in a namespace

You can configure your developer namespace to include more than one pipeline using either of the following methods:

Use a single pipeline running on a container image that includes testing tools and runs a common script to execute tests. This allows you to accommodate multiple workloads based in different languages in the same namespace that use a common make test script, as shown in the following example:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: developer-defined-tekton-pipeline
  labels:
    apps.tanzu.vmware.com/pipeline: test
spec:
  #...
        steps:
          - name: test
            image: <image_that_has_JDK_and_Go>
            script: |-
              cd `mktemp -d`
              wget -qO- $(params.source-url) | tar xvz -m
              make test

Update the pipeline resources to include labels that differentiate between the pipelines. For example, differentiate between Java and Go pipelines by adding labels for Java and Go:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: java-tests
  labels:
    apps.tanzu.vmware.com/pipeline: test
    apps.tanzu.vmware.com/language: java
spec:
  #...
        steps:
          - name: test
            image: gradle
            script: |-
              # ...
              ./mvnw test
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: go-tests
  labels:
    apps.tanzu.vmware.com/pipeline: test
    apps.tanzu.vmware.com/language: go
spec:
  #...
        steps:
          - name: test
            image: golang
            script: |-
              # ...
              go test -v ./...

To match the correct pipeline, you add a testing_pipeline_matching_labels parameter to the workload. For example, if you want to match to the Java pipeline, you have the following workload.yaml:

apiVersion: carto.run/v1alpha1
kind: Workload
metadata:
  name: sample-java-app
  labels:
    apps.tanzu.vmware.com/has-tests: true
    apps.tanzu.vmware.com/workload-type: web
    app.kubernetes.io/part-of: sample-java-app
spec:
  params:
    - name: testing_pipeline_matching_labels
      value:
        apps.tanzu.vmware.com/pipeline: test
        apps.tanzu.vmware.com/language: java
  ...

This matches the workload to the pipeline with the apps.tanzu.vmware.com/language: java label.

Developer Workload

With the Tekton Pipeline object submitted to the same namespace as the one where the Workload will be submitted to, you can submit your Workload.

Regardless of the workflow being targeted (local development or gitops), the Workload configuration details are the same as in Out of the Box Supply Chain Basic, except that you mark the workload with tests enabled using the has-tests label.

For example:

tanzu apps workload create tanzu-java-web-app \
  --git-branch main \
  --git-repo https://github.com/vmware-tanzu/application-accelerator-samples \
  --sub-path tanzu-java-web-app \
  --label apps.tanzu.vmware.com/has-tests=true \
  --label app.kubernetes.io/part-of=tanzu-java-web-app \
  --type web

Create workload:
      1 + |---
      2 + |apiVersion: carto.run/v1alpha1
      3 + |kind: Workload
      4 + |metadata:
      5 + |  labels:
      6 + |    apps.tanzu.vmware.com/workload-type: web
      7 + |    apps.tanzu.vmware.com/has-tests: "true"
      8 + |    app.kubernetes.io/part-of: tanzu-java-web-app
      9 + |  name: tanzu-java-web-app
     10 + |  namespace: default
     11 + |spec:
     12 + |  source:
     13 + |    git:
     14 + |      ref:
     15 + |        branch: main
     16 + |      url: https://github.com/vmware-tanzu/application-accelerator-samples
     17 + |    subPath: tanzu-java-web-app