Tanzu Kubernetes Grid includes binaries for tools that provide in-cluster and shared services to the clusters running in your Tanzu Kubernetes Grid instance. All of the provided binaries and container images are built and signed by VMware.
You can add functionalities to Tanzu Kubernetes clusters by installing extensions to different cluster locations as follows:
Function | Extension | Location | Procedure |
---|---|---|---|
Ingress Control | Contour | Tanzu Kubernetes cluster | Implementing Ingress Control with Contour |
Log Forwarding | Fluent Bit | Tanzu Kubernetes cluster | Implementing Log Forwarding with Fluentbit |
Container Registry | Harbor | Shared Services cluster | Deploy Harbor Registry as a Shared Service |
Monitoring | Prometheus | Tanzu Kubernetes cluster | Implementing Monitoring with Prometheus and Grafana |
Grafana | Tanzu Kubernetes cluster | ||
User Authentication | Dex | Management cluster | Implementing User Authenticaton with Dex and Gangway |
Gangway | Tanzu Kubernetes cluster |
Before you can deploy the Tanzu Kubernetes Grid extensions, you must prepare your bootstrap environment by performing the following tasks.
The Tanzu Kubernetes Grid extension manifests are provided in a separate bundle to the Tanzu Kubernetes Grid CLI and other binaries.
Use either the tar
command or the extraction tool of your choice to unpack the bundle of YAML manifest files for the Tanzu Kubernetes Grid extensions.
tar -xzf tkg-extensions-manifests-v1.2.0-vmware.1.tar.gz
For convenience, unpack the bundle in the same location as the one from which you run tkg
and kubectl
commands.
IMPORTANT: After you unpack the bundle, the extensions files are contained in a folder named tkg-extensions-v1.2.0+vmware.1
. This folder contains subfolders for each type of extension, for example, authentication
, ingress
, registry
, and so on. At the top level of the folder there is an additional subfolder named extensions
. The extensions
folder also contains subfolders for authentication
, ingress
, registry
, and so on. In the procedures to deploy the extensions, take care to run commands from the location provided in the instructions. Commands are usually run from within the extensions
folder.
The Tanzu Kubernetes Grid uses the following tools from the Carvel open-source project to configure and deploy all of the extensions:
Tanzu Kubernetes Grid provides signed binaries for ytt
, kapp
, and kbld
that are bundled with the Tanzu Kubernetes Grid CLI. For information about where to download the CLI bundle, see Install the Tanzu Kubernetes Grid CLI.
To deploy the extensions, you update configuration files with information about your environment. You then use kubectl
to apply preconfigured YAML files that pull data from the updated configuration files to create and update clusters that implement the extensions. The YAML files include calls to ytt
, kapp
, and kbld
commands, so these tools must be present on your bootstrap environment when you deploy the extensions.
Navigate to the location on your bootstrap environment machine where you saved the Tanzu Kubernetes Grid CLI bundle.
The bundles are tkg-darwin-amd64-VERSION
for macOS, tkg-linux-amd64-VERSION
for Linux, and tkg-windows-amd64-VERSION
for Windows.
Rename the YTT binary for your platform to ytt
and make it available to the system:
For macOS and Linux platforms:
/usr/local/bin
: mv ./ytt-linux-amd64-v0.30.0+vmware.1 /usr/local/bin/ytt
mv ./ytt-darwin-amd64-v0.30.0+vmware.1 /usr/local/bin/ytt
ls
command.For Windows platforms:
Program Files\ytt
folder and copy the ytt-windows-amd64-v0.30.0+vmware.1
binary into it.ytt-windows-amd64-v0.30.0+vmware.1
to ytt.exe
.ytt
folder, select Properties > Security, and make sure that your user account has the Full Control permission.env
.Path
row under System variables, and click Edit.ytt
binary.At the command line in a new terminal, run ytt version
to check that the correct version of the binary is properly installed.
You should see information about the installed ytt
version.
ytt version 0.30.0
Rename the Kapp binary for your platform to kapp
and make it available to the system:
For macOS and Linux platforms:
/usr/local/bin
: mv ./kapp-linux-amd64-v0.33.0+vmware.1 /usr/local/bin/kapp
mv ./kapp-darwin-amd64-v0.33.0+vmware.1 /usr/local/bin/kapp
ls
command.For Windows platforms:
Program Files\kapp
folder and copy the kapp-windows-amd64-v0.33.0+vmware.1
binary into it.kapp-windows-amd64-v0.33.0+vmware.1
to kapp.exe
.kapp
folder, select Properties > Security, and make sure that your user account has the Full Control permission.env
.Path
row under System variables, and click Edit.kapp
binary.At the command line in a new terminal, run kapp version
to check that the correct version of the binary is properly installed.
You should see information about the installed kapp
version.
kapp version 0.33.0
Rename the Kbld binary for your platform to kbld
and make it available to the system:
For macOS and Linux platforms:
/usr/local/bin
: mv ./kbld-linux-amd64-v0.24.0+vmware.1 /usr/local/bin/kbld
mv ./kbld-darwin-amd64-v0.24.0+vmware.1 /usr/local/bin/kbld
ls
command.For Windows platforms:
Program Files\kbld
folder and copy the kbld-windows-amd64-v0.24.0+vmware.1
binary into it.kbld-windows-amd64-v0.24.0+vmware.1
to kbld.exe
.kbld
folder, select Properties > Security, and make sure that your user account has the Full Control permission.env
.Path
row under System variables, and click Edit.kbld
binary.At the command line in a new terminal, run kbld version
to check that the correct version of the binary is properly installed.
You should see information about the installed kbld
version.
kbld version 0.24.0
Procedures for installing Tanzu Kubernetes Grid extensions on a cluster, whether management or workload, include installing these components:
extension-manager
cert-manager
To install these components on a cluster, specify the cluster with kubectl config use-context
and then do the following:
Install the Tanzu Mission Control extension manager on the cluster.
The Tanzu Kubernetes Grid extensions and Tanzu Mission Control both use the same extension-manager
service. You must install the extension manager even if you do not intend to use Tanzu Mission Control.
kubectl apply -f tmc-extension-manager.yaml
You should see confirmation that a namespace, resource definitions, a service account, an RBAC role, and a role binding for the extension-manager
service are all created.
namespace/vmware-system-tmc created
customresourcedefinition.apiextensions.k8s.io/agents.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensions.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionresourceowners.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionintegrations.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionconfigs.intents.tmc.cloud.vmware.com created
serviceaccount/extension-manager created
clusterrole.rbac.authorization.k8s.io/extension-manager-role created
clusterrolebinding.rbac.authorization.k8s.io/extension-manager-rolebinding created
service/extension-manager-service created
deployment.apps/extension-manager created
Note: You can disregard any warning messages about deprecated APIs.
Install the Kapp controller on the cluster.
kubectl apply -f kapp-controller.yaml
You should see confirmation that a service account, resource definition, and RBAC role are created for the kapp-controller
service.
serviceaccount/kapp-controller-sa created
customresourcedefinition.apiextensions.k8s.io/apps.kappctrl.k14s.io created
deployment.apps/kapp-controller created
clusterrole.rbac.authorization.k8s.io/kapp-controller-cluster-role created
clusterrolebinding.rbac.authorization.k8s.io/kapp-controller-cluster-role-binding created
Determine whether the extension image you are installing is stored in a private container registry that uses a self-signed certificate:
Look in the extension manifest file in the extensions directory. The file has a name of the form NAME-extension.yaml
, for example harbor-extension.yaml
or contour-extension.yaml
.
In the manifest, check whether the spec.objects.spec.fetch.image.url
field points to your own private registry or a public registry. Private registries typically use self-signed certificates.
If the extension manifest pulls the image from a registry with a self-signed certificate, configure the Kapp controller to trust the registry as follows:
Create a file custom-ca-certs.crt
containing the CA certificate of your private container registry.
Create a ConfigMap with the certificate by running:
kubectl create configmap custom-ca-certs --from-file=custom-ca-certs.crt -n vmware-system-tmc
Back up your kapp-controller.yaml
and replace the kind: Deployment
block with the code below:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kapp-controller
namespace: vmware-system-tmc
spec:
replicas: 1
revisionHistoryLimit: 0
selector:
matchLabels:
app: kapp-controller
template:
metadata:
labels:
app: kapp-controller
spec:
containers:
- args: []
command:
- /kapp-controller
env:
- name: KAPPCTRL_MEM_TMP_DIR
value: /etc/kappctrl-mem-tmp
image: registry.tkg.vmware.run/kapp-controller:v0.9.0_vmware.1
imagePullPolicy: IfNotPresent
name: kapp-controller
resources:
requests:
cpu: 120m
memory: 100Mi
securityContext:
runAsGroup: 2000
runAsUser: 1000
volumeMounts:
- mountPath: /etc/kappctrl-mem-tmp
name: template-fs
- mountPath: /etc/ssl/certs/custom-ca-certs.crt
name: custom-ca-certs
readOnly: true
subPath: custom-ca-certs.crt
securityContext:
fsGroup: 3000
serviceAccount: kapp-controller-sa
volumes:
- emptyDir:
medium: Memory
name: template-fs
- configMap:
name: custom-ca-certs
name: custom-ca-certs
Apply the Kapp controller definition to mount the CA cert file from the ConfigMap:
kubectl apply -f kapp-controller.yaml
For extensions other than Dex or Fluent Bit, deploy cert-manager
, which provides automated certificate management, on the cluster.
kubectl apply -f cert-manager/
Dex and Fluent Bit do not use cert-manager
.
For information about how to upgrade the Tanzu Kubernetes Grid extensions from a previous release, see Upgrade Tanzu Kubernetes Grid Extensions from 1.1.x to 1.2.x.