Manage Stores with your Tanzu Build Service

This topic tells you how to manage Stores with your Tanzu Build Service (commonly known as TBS).

A Store is a cluster-level resource that provides a collection of buildpacks that can be utilized by Builders. Buildpacks are distributed and added to a store in buildpackages, which are docker images containing one or more buildpacks.

Build Service ships with a curated collection of Tanzu buildpacks for Java, Java Native Image, Node.js, Go, .NET Core, Python, Ruby, PHP, Web Servers (NGINX and Apache HTTPD), and Procfile. Detailed documentation about the buildpacks installed with Tanzu Build Service can be found in the Tanzu Buildpacks documentation. It is important to keep these buildpacks up-to-date. Updates to these buildpacks are provided on Tanzu Network.

In addition to supported Tanzu and Paketo buildpacks, custom buildpackages can be uploaded to Build Service stores.

The kp CLI can be used to manage clusterstores. See the kpack-cli help text on GitHub.

$ kp clusterstore
ClusterStore Commands

Usage:
  kp clusterstore [command]

Aliases:
  clusterstore, clusterstores, clstrcsrs, clstrcsr, csrs, csr

Available Commands:
  add         Add buildpackage(s) to cluster store
  create      Create a cluster store
  delete      Delete a cluster store
  list        List cluster stores
  remove      Remove buildpackage(s) from cluster store
  save        Create or update a cluster store
  status      Display cluster store status

Flags:
  -h, --help   help for clusterstore

Note: These docs assume you are using kp CLI v0.10.x with Tanzu Build Service v1.10.x. If a feature is not working, you might need to upgrade your CLI.

Creating buildpacks and buildpackages

For information about creating buildpacks, see Create a buildpack on the Buildpacks.io website.

For information about packaging buildpacks, see Package a buildpack on the Buildpacks.io website.

Note: Only Build Service admins can perform store commands.

Listing ClusterStores

Users can view the existing stores by running:

kp clusterstore list

Creating a ClusterStore

Tanzu Build Service ships with a default store containing all of the supported buildpacks. Users can create additional stores with:

kp clusterstore create <store-name> -b <buildpackage-1> -b <buildpackage-2>

For example:

kp clusterstore create my-store -b my-registry.com/my-buildpackage
kp clusterstore create my-store -b my-registry.com/my-buildpackage -b my-registry.com/my-other-buildpackage
kp clusterstore create my-store -b ../path/to/my-local-buildpackage.cnb

Buildpackages are uploaded to the registry used during installation.

Note: The user must have read access to the source Docker registry and write access to the registry used for installation on the local machine.

Saving a ClusterStore

Users can create or update a ClusterStore using the save command. The kp clusterstore save command is used exactly the same as kp clusterstore create, but it determines whether a clusterstore needs to be created or updated.

kp clusterstore save <store-name> -b <buildpackage-1> -b <buildpackage-2>

Adding buildpackages to a ClusterStore

Users can add multiple buildpackages at a time from a registry or from a file on the local machine.

This command is useful for users that want to only consume certain buildpacks rather than update all dependencies with kp import.

• If using a Docker registry:

  kp clusterstore add <store-name> -b <buildpackage-1> -b <buildpackage-2> ...
  

Note: The user must have read access to the source Docker registry and write access to the registry used for installation on the local machine.

• If using local .cnb buildpackage files created as described in the buildpackages docs:

  kp clusterstore add <store-name> -b <path-to-buildpackage-1>.cnb -b <path-to-buildpackage-2>.cnb ...
  

Adding buildpackages to a ClusterStore from Tanzu Network

Updated versions of all supported Buildpacks are available on Tanzu Network as registry images. Updated Buildpacks are found in the following locations:

• Java
• Java Native Image
• NodeJS
• Go
• .NET Core
• Web Servers
• Procfile
• Python
• Ruby
• PHP

Here is a list of how to update each buildpack included with Tanzu Build Service by default:

kp clusterstore add default registry.tanzu.vmware.com/tanzu-java-buildpack/java:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-java-native-image-buildpack/java-native-image:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-nodejs-buildpack/nodejs:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-go-buildpack/go:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-dotnet-core-buildpack/dotnet-core:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-web-servers-buildpack/web-servers:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-procfile-buildpack/procfile:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-python-buildpack/python:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-ruby-buildpack/ruby:<version>
kp clusterstore add default registry.tanzu.vmware.com/tbs-dependencies/tanzu-buildpacks_php:<version>

If you have installed the lite descriptor, you will want to update your buildpacks using the lite image references below:

kp clusterstore add default registry.tanzu.vmware.com/tanzu-java-buildpack/java-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-java-native-image-buildpack/java-native-image-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-nodejs-buildpack/nodejs-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-go-buildpack/go-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-dotnet-core-buildpack/dotnet-core-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-python-buildpack/python-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-web-servers-buildpack/web-servers-lite:<version>
kp clusterstore add default registry.tanzu.vmware.com/tanzu-ruby-buildpack/ruby-lite:<version>

Offline Adding Buildpackages to a ClusterStore from Tanzu Network

If your Tanzu Build Service installation is in an offline/air-gapped environment, you can update stores with the following offline workflow:

1. Find the latest version of the Dependency Descriptor bundle image (registry.tanzu.vmware.com/tbs-dependencies/full) from the latest release on the Tanzu Build Service Dependencies page on Tanzu Network.

2 Download the Tanzu Cluster Essentials for your operating system and install the following CLIs: - ytt - kbld - imgpkg

    ```console
    docker login registry.tanzu.vmware.com
    ```

- If you are using the `crane` CLI, run:

    ```console
    crane auth login registry.tanzu.vmware.com
    ```

1. Download the dependency images for Tanzu Build Service to your local machine with imgpkg by running:

   imgpkg copy -b registry.tanzu.vmware.com/tbs-dependencies/full:<VERSION> \
     --to-tar=tbs-dependencies.tar
   

2. Move the output file tbs-dependencies.tar to a machine that has access to the “offline” environment.

3. Log in to the image registry used to deploy Tanzu Build Service:

   • If you are using the docker CLI, run:

     docker login <build-service-registry>
     

   • If you are using the crane CLI, run:

     crane auth login <build-service-registry>
     

4. Upload the dependency images to the registry used to deploy Tanzu Build Service by running:

   imgpkg copy --tar=tbs-dependencies.tar \
     --to-repo <IMAGE-REPOSITORY>
   

   Where:

   • IMAGE-REPOSITORY is the repository used to install Tanzu Build Service. This should be the same value as IMAGE-REPOSITORY used in the Installation Steps.

5. Now that dependencies are relocated to the internal registry, you can use the following commands to update the necessary resources:

   imgpkg pull -b <IMAGE-REPOSITORY>:<VERSION> \
     -o /tmp/descriptor-bundle \
     --registry-ca-cert-path <PATH-TO-CA>
   
   kbld -f /tmp/descriptor-bundle/.imgpkg/images.yml \
     -f /tmp/descriptor-bundle/tanzu.descriptor.v1alpha3/descriptor-<VERSION>.yaml \
     | kp import -f -
   

Removing buildpackages from a ClusterStore

Users can remove a buildpackage from a ClusterStore by referencing the buildpackage Id and version.

kp clusterstore remove <store> -b <buildpackage-id>@<buildpackage-version>

For example:

kp clusterstore remove my-store -b [email protected]
kp clusterstore remove my-store -b [email protected] -b [email protected]

The ClusterStore status shows the list of buildpackage ID and version

Get ClusterStore status

Users can use the kp CLI to get details about a store including buildpackages and their buildpacks, as well as meta-buildpacks. Meta-buildpacks are buildpacks that indicate the order that other buildpacks run:

To view the buildpackages in a store, run:

kp clusterstore status <store-name>

For example:

$ kp clusterstore status default

Status:    Ready

BUILDPACKAGE ID                       VERSION    HOMEPAGE
paketo-buildpacks/go                  0.1.3      https://github.com/paketo-buildpacks/go
paketo-buildpacks/procfile            2.0.2      https://github.com/paketo-buildpacks/procfile
paketo-buildpacks/procfile            3.0.0      https://github.com/paketo-buildpacks/procfile
tanzu-buildpacks/dotnet-core          0.0.4
tanzu-buildpacks/dotnet-core          0.0.7
tanzu-buildpacks/dotnet-core          0.0.6
tanzu-buildpacks/go                   1.0.6
tanzu-buildpacks/go                   1.0.7
tanzu-buildpacks/go                   1.0.9
tanzu-buildpacks/go                   1.0.5
tanzu-buildpacks/httpd                0.0.38
tanzu-buildpacks/httpd                0.0.39
tanzu-buildpacks/httpd                0.0.40
tanzu-buildpacks/java                 3.8.0      https://github.com/pivotal-cf/tanzu-java
tanzu-buildpacks/java                 3.5.0      https://github.com/pivotal-cf/tanzu-java
tanzu-buildpacks/java                 4.1.0      https://github.com/pivotal-cf/tanzu-java
tanzu-buildpacks/java                 4.0.0      https://github.com/pivotal-cf/tanzu-java
tanzu-buildpacks/java-native-image    3.6.0      https://github.com/pivotal-cf/tanzu-java-native-image
tanzu-buildpacks/java-native-image    3.9.0      https://github.com/pivotal-cf/tanzu-java-native-image
tanzu-buildpacks/java-native-image    3.4.2      https://github.com/pivotal-cf/tanzu-java-native-image
tanzu-buildpacks/java-native-image    3.10.0     https://github.com/pivotal-cf/tanzu-java-native-image
tanzu-buildpacks/nginx                0.0.48
tanzu-buildpacks/nginx                0.0.46
tanzu-buildpacks/nodejs               1.1.0
tanzu-buildpacks/nodejs               1.2.3
tanzu-buildpacks/nodejs               1.2.2
tanzu-buildpacks/php                  0.0.3
tanzu-buildpacks/php                  0.0.5

To view buildpackages & their individual buildpacks, and display the order of meta-buildpacks, use the --verbose flag:

kp clusterstore status <store-name> --verbose

Migrating buildpacks

Build Service never automatically removes buildpackages from the store unless you explicitly remove them. In this way, users can continue to use older buildpacks until the operator is ready to migrate them.

How you migrate is entirely dependent on the configuration of your Builder resources:

• Builders that do not provide a buildpack version automatically update to the latest buildpack version if it is available.
• Builders that explicitly specify a buildpack version do not update automatically.

With the above in mind, migrating buildpackages in the store is as simple as kp clusterstore adding newer buildpackages and kp clusterstore removeing older buildpackages as necessary.

For fine-grained control over buildpack updates, you can create multiple stores to manage buildpack versions. Then you can point individual builders at the desired store. Each store can be updated as needed without affecting other builders or fanning out large, sweeping changes.

Corresponding kpack resource

All Build Service builders utilize cluster-scoped Store resources.