Enabling Prometheus Metrics Collection

With the Greenplum Streaming Server's out-of-the-box Prometheus integration, you can obtain runtime metrics for a gpss server instance when you enable Prometheus monitoring for the UNIX process.

The GPSS metrics available from Prometheus are:

Name	Description
gpss_jobs_total	The total number of jobs the `gpss` server instance is servicing.
gpss_jobs_running	The number of jobs that are currently in the Running state.
gpss_process_cpu_seconds_total	The total user and system CPU time (in seconds) for the GPSS server process.
gpss_process_open_fds	The number of open file descriptors in the GPSS server instance process.
gpss_process_max_fds	The maximum number of open file descriptors allowed for the GPSS server instance process.
gpss_process_virtual_memory_bytes	The virtual memory size (in bytes) of the GPSS server instance process.
gpss_process_virtual_memory_max_bytes	The maximum virtual memory size (in bytes) of the GPSS server instance process.
gpss_process_resident_memory_bytes	The resident memory size (in bytes) of the GPSS server instance process.
gpss_process_start_time_seconds	The start time of the GPSS server instance process since epoch (in seconds).

Prerequisites

GPSS uses the Prometheus stable HTTP API /api/v1. Before you enable Prometheus integration with GPSS, ensure that:

You are running a Prometheus server compatible with the v1 API. If you are new to Prometheus, you can download and install a Prometheus server as described in the Getting Started topic in the Prometheus documentation.
Connectivity exists between the host on which you run the GPSS server instance and the host on which the Prometheus server is running.

Enabling Prometheus Integration with GPSS

Prometheus collects metrics from monitored processes by scraping HTTP endpoints exposed by the processes. Scraping pulls published statistics that allow you to aggregate and record time series data, or to generate alerts.

To enable Prometheus integration with GPSS, you must:

Identify the host name or IP Address and port number on which Prometheus will pull statistics from the GPSS server instance. Both GPSS and Prometheus require this configuration information.
Add a Monitor property block to the gpss.json GPSS server configuration file that identifies the address and port number from which GPSS will allow Prometheus to pull the server's statistics. For example:
```
Monitor: {
    Prometheus: {
        Listening: "0.0.0.0:5001"
    }
} 
```
The Listening property value specified in this block allows any Prometheus server host to pull from port 5001.
Create a new, or update an existing, YAML-format Prometheus configuration file to include a GPSS scrape target. For example, to configure Prometheus metrics collection for the GPSS server instance running on the host named etlhost that was configured with the Monitor block above, copy/paste the following text into a file named prometheus_gpss_cfg.yml:
```
global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.

# A scrape configuration containing exactly one endpoint to scrape, GPPSS
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries
  # scraped from this config.
  - job_name: 'gpss'

    static_configs:
      - targets: ['etlhost:5001']
```
This configuration instructs Prometheus to pull metrics from port 5001 on etlhost (the GPSS server instance) every fifteen seconds.
Start the Prometheus server, specifying the configuration file that you updated or created for GPSS:
```
$ prometheus --config.file=prometheus_gpss_cfg.yml & 
```
Start the GPSS server:
```
$ gpss --config gpss.json & 
```

Viewing GPSS Metrics

When you run a Prometheus server, metrics for integrated services are available at the default port 9090. To view metrics for all services integrated with Prometheus, navigate to http://<prometheus_host>:9090 in your web browser of choice.

To view the metrics for a specific GPSS server instance, navigate to Status->Targets, and select the GPSS Endpoint of interest.

The default Prometheus web user interface is rather low level. Once enabled, you can use the Prometheus expression browser to generate and run ad-hoc queries on GPSS time series data.

To view graphs of GPSS metrics, enable Grafana integration as described in Grafana Support for Prometheus in the Prometheus documentation.