The Custom Resource Definition (CRD) is a YAML file that defines a Tanzu GemFire cluster called GemFireCluster and specifies its characteristics.

CRD Template

The CRD file specifies fields and values that define the Tanzu GemFire cluster API. The template shown here lists the fields you can set for a Tanzu GemFire cluster. Details are provided after the template.

apiVersion: gemfire.tanzu.vmware.com/v1
kind: GemFireCluster
metadata:
  name: <string>
  namespace: <string>
spec:
  image: <string>
  antiAffinityPolicy: <string>
  metrics:
    emission: <string>
    interval: <string>
  resourceManager:            
    criticalHeapPercentage: <integer>                    
    evictionHeapPercentage: <integer>
  sysctls: <sysctl name/value pairs>
  serialization:            
    pdx:                    
      readSerialized: <boolean>
  locators:
    replicas: <integer>
    resources:
      requests:
        memory: <memory units>
        cpu: <cpu units>
      limits:
        memory: <memory units>
        cpu: <cpu units>
    persistentVolumeClaim:     
      resources:               
        requests:
          storage: <storage units>        
      storageClassName: <string>
    overrides:
      jvmOptions: <list of strings>
      gemfireProperties: <GemFire properties name/value pairs>
      statefulSet:
        spec:
          template:
            metadata:
              name: <string>
              namespace: <string>
              labels: <list of strings>
              annotations: <list of strings>
            spec: <Kubernetes pod template spec>
  servers:
    replicas: <integer>
    resources:
      requests:
        memory: <memory units>
        cpu: <cpu units>
      limits:
        memory: <memory units>
        cpu: <cpu units>
    persistentVolumeClaim:     
      resources:               
        requests:
          storage: <storage units>        
      storageClassName: <string>
    overrides:
      jvmOptions: <list of strings>
      gemfireProperties: <GemFire properties name/value pairs>
      statefulSet:
        spec:
          template:
            metadata:
              name: <string>
              namespace: <string>
              labels: <list of strings>
              annotations: <list of strings>
            spec: <Kubernetes pod template spec>

Fields and Values

The first two fields establish the nature of the CRD and its name. For this release of Tanzu GemFire, these should always be the same:

apiVersion: gemfire.tanzu.vmware.com/v1
kind: GemFireCluster

Instance Metadata

The fields in this section apply metadata to the Tanzu GemFire cluster.

metadata.name

metadata:
  name: <string>

(Required) Sets the name of the GemFireCluster cluster. This value cannot be changed on an existing cluster.

metadata.namespace

metadata:
  namespace: <string>

(Optional) Sets the Kubernetes namespace in which the GemFireCluster cluster will be deployed. If not specified, the current kubectl context’s namespace will be used. This value cannot be changed on an existing cluster.

Cluster Configuration

The fields in this section apply configuration to the entire Tanzu GemFire cluster.

spec.image

spec:
  image: <string>

(Required) Specifies the image to be used by the operator when creating pods for the Tanzu GemFire cluster; therefore, it specifies a cluster version. Updating this value will trigger an upgrade to a new version of the Tanzu GemFire cluster. Downgrades to a previous version are not supported, and will result in a failure of the first locator to restart.

image values:

  • registry.pivotal.io/tanzu-gemfire-for-kubernetes/gemfire-k8s:1.0.0
  • registry.pivotal.io/tanzu-gemfire-for-kubernetes/gemfire-k8s:1.0.1

spec.antiAffinityPolicy

spec:
  antiAffinityPolicy: <string>

(Optional) Configures the anti-affinity policy for the Tanzu GemFire cluster. Each server or locator lives in its own pod. There are implications for data availability when multiple pods reside within a single node. There are three possible case-sensitive string values:

  • "None" allows each node to host multiple server pods and multiple locator pods. This is the policy used if the antiAffinityPolicy is not set.
  • "Cluster" allows each node to host multiple server pods and multiple locator pods, but no more than one of each from a given Tanzu GemFire cluster.
  • "Full" allows each node to host at most one server pod and at most one locator pod.

If there are not enough nodes available to satisfy the anti-affinity policy, pods will remain in the pending state.

spec.metrics.emission

metrics:
  emission: <string>

(Optional) Specifies the quantity of metrics available to be pulled from the Prometheus /metrics endpoint of both locators and servers on port 4321. When not specified, the default value is "Default". There are three possible case-sensitive string values:

  • "Default" makes approximately 200 metrics available for collection at 1-minute intervals.
  • "None" disables metrics availability.
  • "All" makes all approximately 1600 metrics available for collection at 2-second intervals.

spec.metrics.interval

metrics:
  interval: <string>

(Optional) Specifies a time interval at which metrics are available for collection. When set, this value overrides the default interval defined for the metrics.emission field.
An accepted value is a positive integer followed by one of the unit characters: s for seconds, m for minutes, h for hours, d for days, or w for weeks. For example, 4m is four minutes, and 90s is 90 seconds.

spec.resourceManager

  resourceManager:            
    criticalHeapPercentage: <integer>                    
    evictionHeapPercentage: <integer>

(Optional) Sets the eviction-heap-percentage and critical-heap-percentage properties used by the Tanzu GemFire Resource Manager. The default values are 75 (75%) for eviction-heap-percentage and 90 (90%) for critical-heap-percentage. A value of zero disables the feature. Both values may be set independently; if both are set, the eviction-heap-percentage must always be a smaller value than the critical-heap-percentage.

spec.sysctls

sysctls: <sysctl name/value pairs>

(Optional) Configures the sysctls set on GemFireCluster pods. These options will override any options with the same name that are set by default. The configured sysctls must be able to be set independently for each pod on a node (namespaced), and unsafe sysctls must be explicitly allowed. For more information, see Using sysctls in a Kubernetes Cluster. The default setting, net.ipv4.tcp_syncookies: 0, disables TCP SYN cookies. While SYN cookies might be used to protect the system against malicious attacks that flood TCP SYN packets, this feature is not compatible with a stable and busy Tanzu GemFire cluster.

spec.serialization

spec:
  serialization:            
    pdx:                    
      readSerialized: <boolean>

(Optional) When true, enables the readSerialized behavior of cluster members. The default value is true. This value cannot be changed on an existing cluster. When false, some operations will require that domain JAR files are deployed to the Tanzu GemFire cluster.

Member Configuration

The fields in this section apply configuration to the set of locators or the set of servers. These fields are defined under spec.locators and spec.servers.

replicas

replicas: <integer>

(Optional) Sets the quantity of locators and servers in the cluster. The quantity must be greater than zero. The default quantity of locators is one, and the default quanity of servers is two.

resources.requests.memory

resources:
  requests:
    memory: <memory units>

(Optional) Specifies the amount of memory a locator or a server needs. The default value is 1Gi, and the minimum allowed value is 512Mi. This value affects how Kubernetes schedules and runs the pods for a GemFireCluster cluster, and it also determines the quantity of memory that will be specified for garbage collection memory spaces within the JVM. To improve performance, the recommendations in Tuning the JVM’s Garbage Collection Parameters are followed.

In general, the total heap size, Xmx/Xms, is set to 80% of the specified resources.requests.memory, and the new generation space size, NewSize/MaxNewSize, is set to 20% of the total heap size. The non-heap size must be a minimum of 350Mi, and the new generation space size is capped to a maximum value of 4Gi. This table lists how Xmx/Xms and NewSize/MaxNewSize are configured based on the value of resources.requests.memory:

resources.requests.memory Value set for Xmx/Xms Value set for NewSize/MaxNewSize
Less than 512Mi N/A (Error) N/A (Error)
At least 512Mi and less than 1750Mi resources.requests.memory - 350Mi 20% of Xmx/Xms
At least 1750Mi but less than 20Gi 80% of resources.requests.memory 20% of Xmx/Xms
20Gi or more 80% of resources.requests.memory 4Gi

For more information on how Kubernetes schedules and runs pods, see How Pods with resource requests are scheduled and How Pods with resource limits are run. For information about the format of the memory units, see Resource units in Kubernetes.

resources.limits.memory

resources:
  limits:
    memory: <memory units>

(Optional) Specifies the memory limit of each container. This value affects how Kubernetes schedules and runs the pods for a GemFireCluster cluster. If not specified, no limits will be set on the containers. For more information about how Kubernetes schedules and runs pods, see How Pods with resource requests are scheduled and How Pods with resource limits are run. For information about the format of the memory units, see Resource units in Kubernetes.

resources.requests.cpu and resources.limits.cpu

resources:
  requests:
    cpu: <cpu units>
  limits:
    cpu: <cpu units>

(Optional) Specifies the CPU resource needs of each container. The value affects how Kubernetes schedules and runs the pods for a GemFireCluster cluster. If not specified, no CPU requests or limits will be set on the containers. For more information about how Kubernetes schedules and runs pods, see How Pods with resource requests are scheduled and How Pods with resource limits are run. For information about the format of the CPU units, see Resource units in Kubernetes.

persistentVolumeClaim.resources

persistentVolumeClaim:
  resources:
    requests:
      storage: <storage units>

(Optional) Sets the amount of persistent storage requested for each member. The default value for locators is 2G and the default value for servers is 4G. This value cannot be changed on an existing Tanzu GemFire cluster.

persistentVolumeClaim.storageClassName

persistentVolumeClaim:
  storageClassName: <string>

(Optional) The name of the Kubernetes StorageClass to use when allocating the persistent volume claim. To allocate a persistent volume claim, as needed for Tanzu GemFire cluster servers and locators, the Kubernetes cluster must have a defined StorageClass name. If your Kubernetes cluster does not define a default StorageClass name, then your CRD must define one.

overrides.gemfireProperties

overrides:
  gemfireProperties: <GemFire properties name/value pairs>

(Optional) Sets additional GemFire properties that will be applied to locators and servers at start up. These options will override any options with the same name that are set by default.

For example, to enable the Tanzu GemFire Developer REST API, set the start-dev-rest-api property for servers:

overrides:
  gemfireProperties:
    start-dev-rest-api: "true"

overrides.jvmOptions

overrides:
  jvmOptions: <list of strings>

(Optional) If defined, all default JVM options are discarded, and these specified JVM options become the complete list of JVM options used during member startup.

For example:

apiVersion: gemfire.tanzu.vmware.com/v1
kind: GemFireCluster
metadata:
  name: gemfire1
spec:
  image: registry.pivotal.io/tanzu-gemfire-for-kubernetes/gemfire-k8s:1.0.0
  locators:
    overrides:
      jvmOptions:
        - "-Dgemfire.conserve-sockets=false"
        - "-Xlog:gc+age*=trace,safepoint:file=gc.txt:time,uptime:filecount=10,filesize=1M"
        - "-verbose:gc"
        - "-XX:+UseConcMarkSweepGC"
        - "-XX:+UseCMSInitiatingOccupancyOnly"
        - "-XX:CMSInitiatingOccupancyFraction=60"
  servers:
    overrides:
      jvmOptions:
        - "-Dgemfire.conserve-sockets=false"
        - "-verbose:gc"
        - "-Xlog:gc+age*=trace,safepoint:file=gc.txt:time,uptime:filecount=10,filesize=1M"
        - "-XX:+UseConcMarkSweepGC"
        - "-XX:+UseCMSInitiatingOccupancyOnly"
        - "-XX:CMSInitiatingOccupancyFraction=60"

overrides.statefulSet

overrides:
  statefulSet:
    spec:
      template:
        metadata:
          name: <string>
          namespace: <string>
          labels: <list of strings>
          annotations: <list of strings>
        spec: <Kubernetes pod template spec>

(Optional) Represents a set of pods with consistent identities. The StatefulSet consists of a subset of fields that define a set of values that override defaults.

Note: It is possible to override fields that will leave the Tanzu GemFire cluster in a broken state.

This example imposes an override for pod node affinity:

apiVersion: gemfire.tanzu.vmware.com/v1
kind: GemFireCluster
metadata:
  name: gemfire1
spec:
  image: registry.pivotal.io/tanzu-gemfire-for-kubernetes/gemfire-k8s:1.0.0
  locators:
    overrides:
      statefulSet:
        spec:
          template:
            spec:
              affinity:
                nodeAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                  - weight: 1
                    preference:
                      matchExpressions:
                      - key: disktype
                        operator: In
                        values:
                        - mechanical
              containers: []
  servers:
    overrides:
      statefulSet:
        spec:
          template:
            spec:
              affinity:
                nodeAffinity:
                  preferredDuringSchedulingIgnoredDuringExecution:
                  - weight: 1
                    preference:
                      matchExpressions:
                      - key: disktype
                        operator: In
                        values:
                        - ssd
              containers: []
check-circle-line exclamation-circle-line close-line
Scroll to top icon