This topic explains how you can deliver Carvel Packages
, created by the Carvel Package Supply Chains, from a GitOps repository to one or more run clusters using Flux CD and Supply Chain Choreographer.
To use Gitops Delivery with Flux CD, you must complete the following prerequisites:
Workload
that uses a Carvel Package Supply Chain. You must have at least one Carvel Package
generated by this Workload
stored in your GitOps repository. See Carvel Package Supply Chains (beta).As mentioned earlier, to use a build cluster to control the deployment on run clusters, you must install Flux CD kustomize-controller. You can do the installation using the flux cli. Use release v0.41.2.
CautionNewer releases of Kustomize Controller are not compatible with Tanzu Application Platform v1.7.x and later.
To install Flux CD:
# https://fluxcd.io/flux/get-started/
brew install fluxcd/tap/[email protected] (for example, when on macOS)
flux install -n flux-system --components kustomize-controller --version v0.41.2
Each run cluster must have a namespace and ServiceAccount
with the correct permissions to deploy the Carvel Packages
, PackageInstalls
and Kubernetes Secrets
. Create a namespace and ServiceAccount
with the following permissions:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: <run-cluster-ns>
name: app-package-and-pkgi-install-role
rules:
- apiGroups: ["data.packaging.carvel.dev"]
resources: ["packages"]
verbs: ["get", "list", "create", "update", "delete", "patch"]
- apiGroups: ["packaging.carvel.dev"]
resources: ["packageinstalls"]
verbs: ["get", "list", "create", "update", "delete", "patch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "create", "update", "delete", "patch"]
If your run cluster is a Tanzu Application Platform cluster, see Set up developer namespaces to use your installed packages.
If your run cluster is not a Tanzu Application Platform cluster, the ServiceAccount
must also have the following permissions:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: <run-cluster-ns>
name: app-cr-role
rules:
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "create", "update", "delete"]
- apiGroups: [""]
resources: ["configmaps", "services"]
verbs: ["get", "list", "create", "update", "delete"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "create", "update", "delete"]
For each Carvel Package
and each run cluster, you must create a Carvel PackageInstall
and a Secret
. The Carvel PackageInstall
and the Secret
are stored in your GitOps repository and deployed to run clusters by Flux CD.
The following example shows GitOps repository structure after completing this section:
app.default.tap/
packages/
20230321004057.0.0.yaml # Package
staging/
packageinstall.yaml # PackageInstall
params.yaml # Secret
prod/
packageinstall.yaml # PackageInstall
params.yaml # Secret
For each run cluster, create a Secret
that has the values for each Package
parameter. You can see the configurable properties of the Package
by inspecting the Package
CR’s valuesSchema, or in the Carvel Package Supply Chains documentation. Store the Secret
in your GitOps repository at <package_name>/<run_cluster>/params.yaml
.
---
apiVersion: v1
kind: Secret
metadata:
name: app-values
stringData:
values.yaml: |
---
workload_name: app
replicas: 2
hostname: app.mycompany.com
NoteYou must set a value for the
workload_name
parameter. You can skip setting other fields to use the default parameter values.
For each run cluster, create a PackageInstall
. Reference the Secret
you created earlier. Store the PackageInstall
in your GitOps repository at <package_name>/<run_cluster>/packageinstall.yaml
.
---
apiVersion: packaging.carvel.dev/v1alpha1
kind: PackageInstall
metadata:
name: app
spec:
serviceAccountName: <run-cluster-ns-sa> # ServiceAccount on run cluster with permissions to deploy Package, see "Set up run Cluster Namespaces"
packageRef:
refName: app.default.tap # name of the Package
versionSelection:
constraints: 20230321004057.0.0 # version of the Package
values:
- secretRef:
name: app-values # Secret created in previous step
To continuously deploy the latest version of your Package
, set versionSelection.constraints: >=0.0.0
. To revert to a previous version, update the versionSelection.constraints:
field and annotate the PackageInstall:
packaging.carvel.dev/downgradable: ""
See the Carvel documentation.
PackageInstalls
and Secrets
to your GitOps repository.Configure Flux CD on the Build cluster to deploy your Packages
, PackageInstalls
, and Secrets
to each of your run clusters.
Give your Build cluster access to your run clusters. On the Build cluster, for each run cluster, create a Secret
containing the run cluster’s kubeconfig. Create the Kubernetes Secret
in the same namespace as the Kustomization resource:
kubectl create secret generic <run-cluster>-kubeconfig \
-n <build-cluster-ns> \
--from-file=value.yaml=<path-to-run-cluster-kubeconfig>
NoteThe kubeconfig must be self-contained and not rely on binaries, or environment and credential files from the kustomize-controller pod. Kubeconfigs that specify a command to provide client credentials such as
gke-gcloud-auth-plugin
won’t work without a custom per-provider installation of kustomize-controller.
Configure your Build cluster to clone the GitOps repository. On the Build cluster, create the following Flux CD GitRepository
:
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: <package-name>-gitops-repo
namespace: <build-cluster-ns>
spec:
url: # GitOps repo URL
ignore: |
!.git
interval: 30s
ref:
branch: # GitOps repo branch
timeout: 60s
# only required if GitOps repo is private (recommended). The secret below should be present in the same namespace as the GitRepository.
secretRef:
name: <package-name>-gitops-auth
# only required if GitOps repo is private (recommended)
---
apiVersion: v1
kind: Secret
metadata:
name: <package-name>-gitops-auth
namespace: <build-cluster-ns>
type: Opaque
data:
username: # base64 encoded GitHub (or other git remote) username
password: # base64 encoded GitHub (or other git remote) personal access token
Configure your Build cluster to deploy your Package
to the run clusters. For each run cluster, on the Build cluster, create the following Flux CD Kustomization
:
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: <package-name>-<run-cluster>-packages
namespace: <build-cluster-ns>
spec:
sourceRef:
kind: GitRepository
name: <package-name>-gitops-repo
namespace: <build-cluster-ns>
path: "./<package-name>/packages"
interval: 5m
timeout: 5m
prune: true
wait: true
# where to deploy
kubeConfig:
secretRef:
name: <run-cluster>-kubeconfig
targetNamespace: <run-cluster-ns>
serviceAccountName: <run-cluster-ns-sa>
NoteThe Kustomization resource does not accept a
metadata.name
field longer than 63 characters.
Configure your Build cluster to deploy your PackageInstalls
and Secrets
to the run clusters. For each run cluster, on the Build cluster, create the following Flux CD Kustomization
:
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
# for the second run cluster, for example hello-app-prod2-packages
name: <package-name>-<run-cluster>-packageinstalls
namespace: <build-cluster-ns>
spec:
sourceRef:
kind: GitRepository
name: <package-name>-gitops-repo
namespace: <build-cluster-ns>
path: "./<package-name>/<run-cluster>"
interval: 5m
timeout: 5m
prune: true
wait: true
# where to deploy
kubeConfig:
secretRef:
name: <run-cluster>-kubeconfig
targetNamespace: <run-cluster-ns>
serviceAccountName: <run-cluster-ns-sa>
NoteThe Kustomization resource does not accept a
metadata.name
field longer than 63 characters.
To verify your installation:
On your Build cluster, confirm that your Flux CD GitRepository and Kustomizations are reconciling:
kubectl get gitrepositories,kustomizations -A
Target a run cluster. Confirm that all Packages from the GitOps repository are deployed:
kubectl get packages -A
Target a run cluster. Confirm that all PackageInstalls are reconciled:
kubectl get packageinstalls -A
Now you can access your application on each run cluster.