Prometheus Metrics Endpoint

This topic explains how to configure the Prometheus metrics endpoint provided by VMware Tanzu GemFire.

Tanzu GemFire provides a Prometheus endpoint that allows the metrics to be gathered by a metrics collection agent and forwarded to a metrics monitoring platform (such as the VMware Tanzu GemFire Management Console) for further analysis and alerting.

These metrics are disabled by default and must be enabled and configured when starting a Tanzu GemFire member.

---

Configure Tanzu GemFire Metrics

To enable metrics across your Tanzu GemFire cluster you must modify the emission setting of emitted metrics for each member. By default, the metrics endpoint is not started on a member if this property is not specified.

---

Property	Variable	Accepted Values and Example	Default
emission	--J=-Dgemfire.prometheus.metrics.emission=value	Default: Emit approximately 200 metrics. All: Emit all GemFire metrics. None: No metrics will be available on the endpoint. Example: --J=-Dgemfire.prometheus.metrics.emission=All	No Default Set
interval	--J=-Dgemfire.prometheus.metrics.interval=value	A positive integer followed by a unit of time. s: Seconds m: Minutes d: Days w: Weeks Examples: --J=-Dgemfire.prometheus.metrics.interval=4m --J=-Dgemfire.prometheus.metrics.interval=90s	1m (1 minute) Note: If emission is set to All, the default becomes 2m.
host	--J=-Dgemfire.prometheus.metrics.host=value	IP address to bind the Prometheus metrics endpoint. Example: --J=-Dgemfire.prometheus.metrics.host=127.0.0.1	localhost IP address Note: If you have members running on different hosts, you must specify the corresponding public host IP address for each member.
port	--J=-Dgemfire.prometheus.metrics.port=value	Port to bind the Prometheus metrics endpoint. Example: --J=-Dgemfire.prometheus.metrics.port=8001	8001 Note: If you have multiple members, you should specify this property for each member of the GemFire cluster or you will have errors in your log files.

---

Configure GemFire

To enable metrics for a member (a GemFire locator or server), you must provide the following information when you create the member:

• Metrics Port: Specify a unique metrics port mapping for each locator and server. In your gfsh start command, use the --J=-Dgemfire.prometheus.metrics.port option to specify the port and its value. For example, --J=-Dgemfire.prometheus.metrics.port=8001.

• Metrics Emission: Specify the amount of metrics data for each locator and server. In your gfsh start command, use the --J=-Dgemfire.prometheus.metrics.emission option to specify the emission level and its value. For example, --J=-Dgemfire.prometheus.metrics.emission=Default.

Example Startup

1. Start a locator and specify the metrics emission level.

   start locator --name=locator1 --port=10334 --J=-Dgemfire.prometheus.metrics.emission=Default
   

2. After the member has started, locate and record the IP address of the member in the output of the start command. For example, in the sample output below, the IP address is 192.168.50.245:

   Starting a GemFire Locator in /Users/username/my_gemfire/locator1...
   .................
   Locator in /Users/username/my_gemfire/locator1 on 192.168.50.245[10334] as locator1 is currently online.
   Process ID: 4580
   Uptime: 10 seconds
   GemFire Version: 10.1.0
   Java Version: 11.0.17
   Log File: /Users/username/my_gemfire/locator1/locator1.log
   JVM Arguments: -Dgemfire.enable-cluster-configuration=true
   -Dgemfire.load-cluster-configuration-from-dir=false
   -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true
   -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
   Class-Path:
   //Users/username/my_gemfire/locator1/gemfire-bootstrap-10.0.1.jar
   
   Successfully connected to: JMX Manager [host=192.168.50.245, port=1099]
   
   Cluster configuration service is up and running.
   

3. Verify that the metrics module is properly configured by visiting the following URL:

   http://HOSTNAME:8001/metrics
   

   Where HOSTNAME is the IP address of the member that you recorded in a previous step.

   For example, using the example information above, the URL would be http://192.168.50.245:8001/metrics.

   The output should resemble the following:

   # HELP gemfire_replyWaitTime
   # TYPE gemfire_replyWaitTime gauge
   gemfire_replyWaitTime{category="DistributionStats",instance="distributionStats",member="192.168.
   129.137(locator1:76435:locator)<ec><v0>:41000",} 0.0
   # HELP gemfire_loadsCompleted
   # TYPE gemfire_loadsCompleted gauge
   gemfire_loadsCompleted{category="CachePerfStats",instance="RegionStats-managementRegionStats",member="192.168.129.137(locator1:76435:locator)<ec><v0>:41000",} 0.0
   gemfire_loadsCompleted{category="CachePerfStats",instance="cachePerfStats",member="192.168.129.137(locator1:76435:locator)<ec><v0>:41000",} 0.0
   ...
   

4. Repeat steps 1-3 for each additional member for the cluster, changing the metrics port for each member.