This topic describes how to install or uninstall the VMware Tanzu GemFire Operator when using VMware Tanzu GemFire for Kubernetes.
This version of VMware Tanzu GemFire for Kubernetes defines a Tanzu GemFire Operator to use when creating a Tanzu GemFire cluster. See Operator pattern in the Kubernetes documentation.
A Tanzu GemFire Operator uses Helm charts and Docker images to define the Tanzu GemFire cluster.
For prerequisites and supported platforms, see Prerequisites and Supported Platforms.
Before installing the Tanzu GemFire Operator when using VMware Tanzu GemFire for Kubernetes, you must install cert-manager. You cannot install cert-manager retroactively.
To install cert-manager, run the following on a command line:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.0/cert-manager.yaml
For more information about cert-manager, see the cert-manager documentation.
To install the Operator:
Verify your current Kubernetes cluster:
kubectl config current-context
Create a namespace to be used for the Tanzu GemFire cluster operator and substitute its name in the following examples in place of NAMESPACE-NAME
:
kubectl create namespace NAMESPACE-NAME
Create an image pull secret for the Kubernetes cluster namespace. Set the user name (USERNAME
) and password (PASSWD
) credentials to values that have permission to access registry.packages.broadcom.com. Token for repository access must be retrieved from the Tanzu GemFire product page (Login Required).
kubectl create secret docker-registry image-pull-secret --namespace=NAMESPACE-NAME --docker-server=registry.packages.broadcom.com --docker-username='USERNAME' --docker-password='PASSWD'
Where your USERNAME
and user’s PASSWD
are set.
Surround both the USERNAME
and the PASSWD
by single quote marks to ensure that special characters within those values are handled correctly.
If you are using Tanzu Kubernetes Grid (TKG) as your Kubernetes platform, create a role binding within the NAMESPACE-NAME
namespace, such that pods can be created under the pod security policy that will be configured for your TKG platform:
kubectl create rolebinding psp-gemfire --namespace=NAMESPACE-NAME \
--clusterrole=psp:vmware-system-privileged --serviceaccount=NAMESPACE-NAME:default
Complete the operator installation by installing with Helm or installing with Carvel:
Authenticate Helm with the Broadcom Support registry:
helm registry login -u 'USERNAME' registry.packages.broadcom.com
Password:
Login succeeded
Where USERNAME
is your user name for Broadcom Support and Password is the access token from the Tanzu GemFire product page
Use Helm to install the Custom Resource Definition (CRD) and operator. If cert-manager is installed in a namespace other than cert-manager
, specify that namespace by appending --set certManagerNamespace=<namespace>
to the helm install
command.
helm install gemfire-crd oci://registry.packages.broadcom.com/tanzu-gemfire-for-kubernetes/gemfire-crd --version 2.4.0 --namespace NAMESPACE-NAME --set operatorReleaseName=gemfire-operator
helm install gemfire-operator oci://registry.packages.broadcom.com/tanzu-gemfire-for-kubernetes/gemfire-operator --version 2.4.0 --namespace NAMESPACE-NAME
(Optionally) If you use a custom repository for the VMware Tanzu GemFire for Kubernetes controller image, set the value of controllerImage
when installing the operator. If authorization to an image registry is provided using a kubernetes secret, set the value of imagePullSecretName
to the name of the secret. This might be necessary when access to registry.packages.broadcom.com is unavailable:
helm install gemfire-operator oci://registry.packages.broadcom.com/tanzu-gemfire-for-kubernetes/gemfire-operator --version 2.4.0 --namespace NAMESPACE-NAME --set controllerImage=CUSTOM-REGISTRY-SERVER/gemfire-controller:2.4.0 --set imagePullSecretName=IMAGE-PULL-SECRET-NAME
To perform an offline installation using the Helm chart, ensure that the gemfire-crd
and gemfire-operator
Helm charts, along with the following container images, are hosted on an accessible registry:
Image Name | Required by |
---|---|
gemfire-controller:2.4.0 | gemfire-operator |
vmware-gemfire | GemFire cluster member pods |
cert-manager-cainjector:v1.14.4 | cert-manager |
cert-manager-webhook:v1.14.4 | cert-manager |
cert-manager-controller:v1.14.4 | cert-manager |
NoteWhen installing the gemfire-operator chart, you must set the
controllerImage
value to point to the correct registry location forgemfire-controller:2.4.0
. If you need to authorize access to an image registry using a Kubernetes secret, set theimagePullSecretName
to the name of that secret. This is essential if access to registry.packages.broadcom.com is unavailable.
Use the following command to install:
helm install gemfire-operator oci://registry.packages.broadcom.com/tanzu-gemfire-for-kubernetes/gemfire-operator --version 2.4.0 --namespace NAMESPACE-NAME --set controllerImage=CUSTOM-REGISTRY-SERVER/gemfire-controller:2.4.0 --set imagePullSecretName=IMAGE-PULL-SECRET-NAME
Modify the cert-manager.yaml
file to reference the container images in your registry.
Optional: If your GemFire cluster is configured for off-platform access using a gateway provisioner, ensure that the gateway provisioner’s container image is also hosted on your registry. For example, if you are using Contour as the gateway provider, ensure the contour:v1.29.1
image is accessible and modify the contour-gateway-provisioner.yaml
file to point to the correct registry location for contour:v1.29.1
.
Use imgpkg to fetch the operator Carvel bundle:
imgpkg pull -b registry.packages.broadcom.com/tanzu-gemfire-for-kubernetes/gemfire-for-kubernetes-carvel-bundle:2.4.0 -o workspace/tg4k8s_carvel_bundle
cd workspace/tg4k8s_carvel_bundle && ls
.
.imgpkg
..
certificates.yaml
operator.yaml
functions.lib.yml
values.yaml
Modify values.yaml inside the Carvel bundle with your desired values:
namespace: The kubernetes namespace where the operator will be installed
name: The value of this field is prepended to the names of all resources
certManagerNamespace: The kubernetes namespace where cert-manager is installed
imagePullSecretName: The name of the kubernetes secret that contains credentials to access an image registry
tlsSecretName: The name of the kubernetes secret for TLS
registry:
server: Address to an image registry
username: Required if using a private registry
password: Required if using a private registry
Chain the Carvel tools to complete the operator manifest and deploy the application:
ytt -f operator.yaml -f certificates.yaml -f values.yaml -f functions.lib.yml | kbld -f- | kapp -y deploy -a gemfire-operator -f-
Complete the following steps to verify that the Tanzu GemFire CRD and Operator are successfully deployed:
Run:
kubectl get crd gemfireclusters.gemfire.vmware.com
When the Tanzu GemFire CRD is deployed, output will look similar to the following:
NAME CREATED AT
gemfireclusters.gemfire.vmware.com yyyy-MM-ddTHH:mm:ssZ
Run:
kubectl get pods --namespace NAMESPACE-NAME
When the Tanzu GemFire Operator is deployed, output will look similar to the following:
NAME READY STATUS RESTARTS AGE
gemfire-operator-controller-manager-xxxxxxxxx-xxxxx 1/1 Running 0 ##m
If the status ErrImagePull
or ImagePullBackOff
is encountered, verify the following:
Delete all Tanzu GemFire clusters prior to uninstalling the operator, as described in Delete a Tanzu GemFire Cluster.
Uninstall the Tanzu GemFire Operator and CRD with Helm or with Carvel:
Run:
helm uninstall gemfire-operator --namespace NAMESPACE-NAME
helm uninstall gemfire-crd --namespace NAMESPACE-NAME
Run:
kapp -y delete -a gemfire-operator
kubectl delete namespace NAMESPACE-NAME
This command sequence also destroys all Tanzu GemFire clusters created with the Tanzu GemFire Operator. However, it may not complete the operation, leaving pods forever in the terminating state. Any remaining pods may interfere with future installations, so destroy the pods.
Discover any remaining pods:
kubectl get pods
NAME READY STATUS RESTARTS AGE
gemfire1-server-0 1/1 Terminating 0 29m
gemfire1-server-1 1/1 Terminating 0 29m
Delete each remaining pod with the following two-command sequence:
kubectl patch pod POD-NAME -p '{"metadata":{"finalizers":null}}'
kubectl delete pod POD-NAME --grace-period=0 --force
Where POD-NAME
is the pod’s name, as listed in the kubectl get pods
command.
Remove any persistent volume claims as instructed in Delete a Tanzu GemFire Cluster.