The Custom Resource Definition (CRD) is a YAML file that defines a Tanzu GemFire cluster called GemFireCluster and specifies its characteristics.

CRD Template

The CRD file specifies fields and values that define the Tanzu GemFire cluster API. The template shown here lists the fields you can set for a Tanzu GemFire cluster. Details are provided after the template.

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <string>
  namespace: <string>
spec:
  serviceAccountName: <string>
  image: <string>
  imagePullSecrets: <list of references>
  antiAffinityPolicy: <string>
  metrics:
    emission: <string>
    interval: <string>
  resourceManager:            
    criticalHeapPercentage: <integer>                    
    evictionHeapPercentage: <integer>
  sysctls: <list of sysctls name/value pairs>
  serialization:            
    pdx:                    
      readSerialized: <boolean>
  security.tls
    secretName: <string>
    clientAuthenticationRequired: <boolean>
  security:
    tls:
      secretName: <string>
      clientAuthenticationRequired: <boolean>
    mgmtSvcCredentialsSecretName: <string>
  locators:
    replicas: <integer>
    labels: <list of key/value pairs>
    annotations: <list of key/value pairs>
    libraries: <list of objects>
      - name: <string>
        container:
          image: <string>
          path: <string>
          imagePullSecretRef: <reference>
    resources:
      requests:
        memory: <memory units>
        cpu: <cpu units>
      limits:
        memory: <memory units>
        cpu: <cpu units>
    persistentVolumeClaim:     
      resources:               
        requests:
          storage: <storage units>        
      storageClassName: <string>
    overrides:
      jvmOptions: <list of strings>
      gemFireProperties: <list of GemFireProperties name/value pairs>
      statefulSet:
        spec:
          template:
            metadata:
              name: <string>
              namespace: <string>
              labels: <list of strings>
              annotations: <list of strings>
            spec: <Kubernetes pod template spec>
  servers:
    replicas: <integer>
    labels: <list of key/value pairs>
    annotations: <list of key/value pairs>
    libraries: <list of objects>
      - name: <string>
        container:
          image: <string>
          path: <string>
          imagePullSecretRef: <reference>
    resources:
      requests:
        memory: <memory units>
        cpu: <cpu units>
      limits:
        memory: <memory units>
        cpu: <cpu units>
    persistentVolumeClaim:     
      resources:               
        requests:
          storage: <storage units>        
      storageClassName: <string>
    overrides:
      jvmOptions: <list of strings>
      gemFireProperties: <list of GemFireProperties name/value pairs>
      statefulSet:
        spec:
          template:
            metadata:
              name: <string>
              namespace: <string>
              labels: <list of strings>
              annotations: <list of strings>
            spec: <Kubernetes pod template spec>

Fields and Values

The first two fields establish the nature of the CRD and its name. For this release of Tanzu GemFire, these should always be the same:

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster

Instance Metadata

The fields in this section apply metadata to the Tanzu GemFire cluster.

metadata.name

metadata:
  name: <string>

(Required) Sets the name of the GemFireCluster cluster. This value cannot be changed on an existing cluster.

metadata.namespace

metadata:
  namespace: <string>

(Optional) Sets the Kubernetes namespace in which the GemFireCluster cluster will be deployed. If not specified, the current kubectl context’s namespace will be used. This value cannot be changed on an existing cluster.

Cluster Configuration

The fields in this section apply configuration to the entire Tanzu GemFire cluster.

spec.serviceAccountName

spec:
  serviceAccountName: <string>

(Optional) Specifies the Kubernetes Service Account to be configured by the gemfire-operator within all pods for the Tanzu GemFire cluster. When not specified, the default Service Account for the namespace is automatically configured by Kubernetes itself.

spec.image

spec:
  image: <string>

(Required) Specifies the image to be used by the gemfire-operator when creating pods for the Tanzu GemFire cluster; therefore, it specifies a cluster version. Updating this value will trigger an upgrade to a new version of the Tanzu GemFire cluster. Downgrades to a previous version are not supported, and will result in a failure of the first locator to restart.

spec.imagePullSecrets

spec:
  imagePullSecrets: <list of references>

(Optional) Specifies the list of names for image pull secrets in this Tanzu GemFire cluster’s namespace. Required if more than one registry holds images that are needed for creating a Tanzu GemFire cluster. When not specified, the name of the single image pull secret used during the application of the CRD for creating a Tanzu GemFire cluster is image-pull-secret.

Include each image pull secret’s name in this list of references, as in this example with two image pull secrets:

spec:
  imagePullSecrets:
  - name: "secret-1"
  - name: "secret-2"

spec.antiAffinityPolicy

spec:
  antiAffinityPolicy: <string>

(Optional) Configures the anti-affinity policy for the Tanzu GemFire cluster. There are implications for data availability when multiple pods reside within a single node. There are three possible case-sensitive string values:

  • None allows each node to host multiple server pods and multiple locator pods. This is the policy used if the antiAffinityPolicy is not set.
  • Cluster allows each node to host multiple server pods and multiple locator pods, but no more than one of each from a given Tanzu GemFire cluster.
  • Full allows each node to host at most one server pod and at most one locator pod.

If there are not enough nodes available to satisfy the anti-affinity policy, pods will remain in the pending state.

spec.metrics.emission

metrics:
  emission: <string>

(Optional) Specifies the quantity of metrics available to be pulled from the Prometheus /metrics endpoint for both locators and servers on port 4321. When not specified, the default value is Default. There are three possible case-sensitive string values:

  • None disables metrics availability.
  • Default makes approximately 200 metrics available for collection at 1-minute intervals.
  • All makes all approximately 1600 metrics available for collection at 2-second intervals.

spec.metrics.interval

metrics:
  interval: <string>

(Optional) Specifies a time interval at which metrics are available for collection. When set, this value overrides the default interval defined for the metrics.emission field. An accepted value is a positive integer followed by one of the unit characters: s for seconds, m for minutes, h for hours, d for days, or w for weeks.

For example, 4m is four minutes, and 90s is 90 seconds.

spec.resourceManager

  resourceManager:            
    criticalHeapPercentage: <integer>                    
    evictionHeapPercentage: <integer>

(Optional) Sets the eviction-heap-percentage and critical-heap-percentage properties used by the Tanzu GemFire Resource Manager . The default values are 75 (75%) for eviction-heap-percentage and 90 (90%) for critical-heap-percentage. A value of zero disables the feature. Both values may be set independently; if both are set, the eviction-heap-percentage must always be a smaller value than the critical-heap-percentage.

spec.sysctls

sysctls: <list of sysctls name/value pairs>

(Optional) Configures the sysctls set on GemFireCluster pods. These options will override any options with the same name that are set by default. The configured sysctls must be able to be set independently for each pod on a node (namespaced), and unsafe sysctls must be explicitly allowed. For more information, see Using sysctls in a Kubernetes Cluster. The default setting (shown below) disables TCP SYN cookies. While SYN cookies might be used to protect the system against malicious attacks that flood TCP SYN packets, this feature is not compatible with a stable and busy Tanzu GemFire cluster.

spec:
  sysctls:
    - name: "net.ipv4.tcp_syncookies"
      value: "0"

spec.serialization

spec:
  serialization:            
    pdx:                    
      readSerialized: <boolean>

(Optional) When true, enables the readSerialized behavior of cluster members. The default value is true. This value cannot be changed on an existing cluster. When false, some operations will require that domain JAR files are deployed to the Tanzu GemFire cluster.

spec.security.tls

spec:
  security:
    tls:
      secretName: <string>
      clientAuthenticationRequired: <boolean>

(Optional) Configures TLS for the Tanzu GemFire cluster. The default value is

spec:
  security:
    tls:
      secretName: <CLUSTER-NAME>-cert
      clientAuthenticationRequired: false

where CLUSTER-NAME is the name of the Tanzu GemFire cluster (value used for field metadata.name).

To disable TLS, set the TLS configuration to the empty struct as in this example:

spec:
  security:
    tls: {}

spec.security.tls.secretName

spec:
  security:
    tls:
      secretName: <string>

(Required if TLS configuration is not empty) Sets the name of the Secret containing tls.crt, tls.key, ca.crt , keystore.p12, truststore.p12, and password keys. If a Secret with this name does not exist in the same namespace as the Tanzu GemFire cluster, a new Secret will be created with an auto-generated certificate and trust anchor. This value cannot be changed on an existing cluster.

spec.security.tls.clientAuthenticationRequired

spec:
  security:
    tls:
      clientAuthenticationRequired: <boolean>

(Required if TLS configuration is not empty) Turns on or off TLS client authentication on the Tanzu GemFire cluster. This value cannot be changed on an existing cluster.

spec.security.mgmtSvcCredentialsSecretName

spec:
  security:
    mgmtSvcCredentialsSecretName: <string>

(Optional) Configures the name of the Secret containing the credentials that will be used by the gemfire-operator and lifecycle hooks when connecting to the Tanzu GemFire cluster. The field is mandatory when Authentication and Authorization are enabled for the Tanzu GemFire cluster. Since both basic authentication and token based authentication are supported, the Secret must have either the username and password keys, or the token key.

Member Configuration

The fields in this section apply configuration to the set of locators or the set of servers. These fields are defined under spec.locators and spec.servers.

replicas

replicas: <integer>

(Optional) Sets the quantity of locators and servers in the cluster. The quantity must be greater than zero. The default quantity of locators is one, and the default quantity of servers is two. The amount of locators (spec.locators.replicas) can not be changed on an existing cluster.

labels

labels: <list of key/value pairs>

(Optional) Adds specified labels to locators and servers in the cluster without risk of overriding others. There is no default defined.

For example:

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <CLUSTER-NAME>
spec:
  image: <IMAGE-NAME>
  locators:
    labels:
      environment: production
  servers:
    labels:
      environment: production
      tier: premium

annotations

annotations: <list of key/value pairs>

(Optional) Adds specified annotations to locators and servers in the cluster, allowing for any existing annotations to be overridden. There is no default defined.

For example:

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <CLUSTER-NAME>
spec:
  image: <IMAGE-NAME>
  locators:
    annotations:
      issue-tracking: "https://my-issue-tracking-system/"
  servers:
    annotations:
      issue-tracking: "https://my-issue-tracking-system/"
      last-modified: 2021-07-01

libraries

libraries: <list of objects>
  - name: <string>
    container:
      image: <string>
      path: <string>
      imagePullSecretRef: <reference>

(Optional) Specifies a list of custom jars to be added to the member’s classpath.

The table below describes the fields in more detail.

Field Required Description
name Yes Arbitrary ID, must follow the syntax rules for a DNS name and be unique per member
image Yes Image reference, from a public or private image registry. The image must have the Bourne-again SHell (bash) installed and executable through /bin/bash
path Yes Location, within the image, from which to copy the libraries. Can be a single file or an expression resolvable by the Bourne-again SHell (bash)
imagePullSecretRef No Secret reference to be used as the imagePullSecret when pulling the image. Required only when the image is hosted in a private registry that requires authentication

For example:

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <CLUSTER-NAME>
spec:
  image: <IMAGE-NAME>
  locators:
    libraries:
      - name: my-custom-security-manager
        container:
          image: my.company.com/myImage:1.0.0
          path: "/build/libs/*.jar"
          imagePullSecretRef:
            name: pull-secret-name
  servers:
    libraries:
      - name: my-custom-security-manager
        container:
          image: my.company.com/myImage:1.0.0
          path: "/build/libs/*.jar"
          imagePullSecretRef:
            name: pull-secret-name

resources.requests.memory

resources:
  requests:
    memory: <memory units>

(Optional) Specifies the amount of memory a locator or a server needs. The default and minimum allowed value is 1Gi. This value affects how Kubernetes schedules and runs the pods for a Tanzu GemFire cluster, and it also determines the quantity of memory that will be specified for garbage collection memory spaces within the JVM. To improve performance, follow the recommendations in Tuning the JVM’s Garbage Collection Parameters in Managing Heap and Off-heap Memory in the VMware Tanzu GemFire documentation.

In general, the total heap size, Xmx/Xms, is set to 80% of the specified resources.requests.memory, and the new generation space size, NewSize/MaxNewSize, is set to 20% of the total heap size. The non-heap size must be a minimum of 350Mi, and the new generation space size is capped to a maximum value of 4Gi. This table lists how Xmx/Xms and NewSize/MaxNewSize are configured based on the value of resources.requests.memory:

resources.requests.memory Value set for Xmx/Xms Value set for NewSize/MaxNewSize
Less than 1Gi N/A (Error) N/A (Error)
At least 1750Mi but less than 20Gi 80% of resources.requests.memory 20% of Xmx/Xms
20Gi or more 80% of resources.requests.memory 4Gi

For more information on how Kubernetes schedules and runs pods, see How Pods with resource requests are scheduled and How Pods with resource limits are run. For information about the format of the memory units, see Resource units in Kubernetes.

resources.limits.memory

resources:
  limits:
    memory: <memory units>

(Optional) Specifies the memory limit of each container. This value affects how Kubernetes schedules and runs pods for a Tanzu GemFire cluster. If not specified, no limits will be set on the containers. For more information about how Kubernetes schedules and runs pods, see How Pods with resource requests are scheduled and How Pods with resource limits are run. For information about the format of the memory units, see Resource units in Kubernetes.

resources.requests.cpu and resources.limits.cpu

resources:
  requests:
    cpu: <cpu units>
  limits:
    cpu: <cpu units>

(Optional) Specifies the CPU resource needs of each container. The value affects how Kubernetes schedules and runs the pods for a Tanzu GemFire cluster. If not specified, no CPU requests or limits will be set on the containers. For more information about how Kubernetes schedules and runs pods, see How Pods with resource requests are scheduled and How Pods with resource limits are run. For information about the format of the CPU units, see Resource units in Kubernetes.

persistentVolumeClaim.resources

persistentVolumeClaim:
  resources:
    requests:
      storage: <storage units>

(Optional) Sets the amount of persistent storage requested for each member. The default value for locators is 2G and the default value for servers is 4G. This value cannot be changed on an existing Tanzu GemFire cluster.

persistentVolumeClaim.storageClassName

persistentVolumeClaim:
  storageClassName: <string>

(Optional) The name of the Kubernetes StorageClass to use when allocating the persistent volume claim. To allocate a persistent volume claim, as needed for Tanzu GemFire cluster servers and locators, the Kubernetes cluster must have a defined StorageClass name. If your Kubernetes cluster does not define a default StorageClass name, then your CRD must define one.

overrides.gemFireProperties

overrides:
  gemFireProperties: <list of GemFireProperties name/value pairs>

(Optional) Sets additional GemFire properties that will be applied to locators and servers at start up. These options will override any options with the same name that are set by default.

For example, to enable the Tanzu GemFire Developer REST API, set the start-dev-rest-api property for servers:

overrides:
  gemFireProperties:
    - name: "start-dev-rest-api"
      value: "true"

Note: The following properties cannot be overridden as they are automatically configured when Transport Layer Security is enabled; the deployment will be rejected when trying do so:

  • ssl-keystore
  • ssl-keystore-password
  • ssl-truststore
  • ssl-truststore-password
  • ssl-enabled-components
  • ssl-require-authentication
  • ssl-web-require-authentication
  • ssl-endpoint-identification-enabled

Note: The following properties cannot be overridden on locators as they are automatically configured and required by the gemfire-operator; the deployment will be rejected when trying do so:

  • http-service-port
  • http-service-bind-address
  • jmx-manager-start
  • jmx-manager-port
  • jmx-manager-hostname-for-clients

overrides.jvmOptions

overrides:
  jvmOptions: <list of strings>

(Optional) If defined, the specified JVM options will be appended to the end of the JVM options list used during member startup. If multiple values for the same JVM option are specified in the list, the last value will take precedence over the former value(s).

This example overrides the initial amount of memory allocated:

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <CLUSTER-NAME>
spec:
  image: <IMAGE-NAME>
  locators:
    overrides:
      jvmOptions:
        - "-Xms170m"
  servers:
    overrides:
      jvmOptions:
        - "-Xms170m"

Note: Using this field to configure Tanzu GemFire properties is not recommended; use the overrides.gemFireProperties field instead.

overrides.statefulSet

overrides:
  statefulSet:
    spec:
      template:
        metadata:
          name: <string>
          namespace: <string>
          labels: <list of strings>
          annotations: <list of strings>
        spec: <Kubernetes pod template spec>

(Optional) Represents a set of pods with consistent identities. The StatefulSet consists of a subset of fields that define a set of values that override defaults.

This example imposes an override for pod node affinity:

apiVersion: gemfire.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <CLUSTER-NAME>
spec:
  image: <IMAGE-NAME>
  locators:
    overrides:
      statefulSet:
        spec:
          template:
            spec:
              affinity:
                nodeAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                  - weight: 1
                    preference:
                      matchExpressions:
                      - key: disktype
                        operator: In
                        values:
                        - mechanical
              containers: []
  servers:
    overrides:
      statefulSet:
        spec:
          template:
            spec:
              affinity:
                nodeAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                  - weight: 1
                    preference:
                      matchExpressions:
                      - key: disktype
                        operator: In
                        values:
                        - ssd
              containers: []

Note: It is possible to override fields in a manner that leaves the Tanzu GemFire cluster in a broken state.

Note: The following volume names cannot be used when adding extra Kubernetes Volumes to the overridden statefulSet; the deployment will be rejected when trying do so:

  • init
  • data
  • certs
  • security
  • extensions

Note: The following paths cannot be used when mounting extra resources into locator or server container; the deployment will be rejected when trying do so:

  • “/init”
  • “/data”
  • “/certs”
  • “/security”
  • “/gemfire/extensions”
check-circle-line exclamation-circle-line close-line
Scroll to top icon