gpss.json

Greenplum Streaming Server configuration file.

Synopsis

{
    "ListenAddress": {
        "Host": "<gpss_host>",
        "Port": <gpss_portnum> [,
        "DebugPort": <gpss_debug_portnum>] [,
        "Certificate": {
            "CertFile": "<certfile_path>",
            "KeyFile": "<keyfile_path>",
            "CAFile": "<CAfile_path>",
            "MinTLSVersion": "<min_version>"
        }]
    },
    "Gpfdist": {
        "Host": "<gpfdist_host>",
        "Port": <gpfdist_portnum> [,
        "ReuseTables": <bool_value> ][,
        "Certificate": {
            "CertFile": "<certfile_path>",
            "KeyFile": "<keyfile_path>",
            "CAFile": "<CAfile_path>",
            "MinTLSVersion": "<min_version>",
            "DBClientShared": <bool_value>
        }] [,
        "BindAddress": <bind_addr> ]
    }[,
    "Shadow": {
        "Key": "<passwd_shadow_key>",
    }] [,
    "Authentication": {
        "Username": "<client_auth_username>",
        "Password": "SHADOW:<shadowed_passwd_string>"
    }] [,
    "JobStore": {
        "File": {
            "Directory": "<jobstore_dir>"
        }
    }] [,
    "Logging": {
        "BackendLevel": "<level>",
        "FrontendLevel": "<level>",
        "SplitByJob": "<level>",
        "Rotate": "<policy_period>"
    }] [,
    "Monitor": {
        "Prometheus": {
            "Listening": "<prom_addr>:<prom_port>"
        }
    }] [,
    "KeepAlive": {
        "MaxConnectionIdle": "<duration>",
        "Time": "<duration>",
        "Timeout": "<duration>",
        "MinTime": "<duration>",
        "PermitWithoutStream": <bool_value>
    }]
}

Description

You specify runtime configuration properties for a gpss service instance in a JSON-formatted configuration file. Run properties for GPSS include gpss and gpfdist service hosts, addresses, port numbers, and optional encryption certificates and key files. You can specify a directory in which GPSS stores job status information, addresses, port numbers, and optional encryption certificates and key files.

You can configure the log level of messages that GPSS commands write to stdout and to log files. Other logging-related properties allow you to configure a log rotation period and separate log files per job. You can also configure client-to-server authentication for GPSS. You can also specify a shadow key that GPSS uses to encode and decode the Greenplum Database and client authentication passwords.

This reference page uses the name gpss.json to refer to this file; you may choose your own name for the file.

Keywords and Values

GPSS ListenAddress Options

Host: gpss_host

The host name or IP address on which GPSS listens for client connections. The default host is 127.0.0.1.

Port: gpss_portnum

The port number on which the gpss service instance listens. The default port number is 5000.

DebugPort: gpss_debug_portnum

The port number on which gpss starts a debug server on the local host ( 127.0.0.1). When you specify this configuration option, gpss makes debug information including the call stack and performance statistics available from the debug server.

Certificates:

When you specify certificates, GPSS uses the SSL certificates to authenticate both the client connection and the connection to Greenplum Database.

CertFile: certfile_path: File system path to the server certificate.
KeyFile: keyfile_path: File system path to the server key file.
CAFile: CAfile_path: File system path to the Certificate Authority file. The CAfile_path must contain the entire Certificate Authority chain.
MinTLSVersion: min_version: The minimum transport layer security (TLS) version that GPSS requests on the connection. The default value is 1.0. GPSS supports minimum TLS versions of 1.0, 1.1, 1.2, and 1.3.

Gpfdist Options

GPSS implements the gpfdist protocol. The gpss.json file exposes configuration options that you can use to identify the service location and bind options.

Host: gpfdist_host

The gpfdist service host name or IP address that GPSS sets in the external table LOCATION clause. This hostname or IP address must be reachable from each Greenplum Database segment host. The default value is the fully qualified distinguished name of the host on which GPSS is running.

Port: gpfdist_portnum

The gpfdist service port number. The default port number is 8080.

ReuseTables: bool_value

A boolean that identifies whether or not GPSS should reuse an external table when the job associated with a load operation is restarted. The default value is true, reuse the external table. When you reuse external tables, GPSS generates the external table name using a hash of various server and load configuration property values.

If you choose not to reuse external tables, GPSS drops the external table associated with a load operation (if one exists) and creates a new external table when you (re)start the job. If you do not reuse external tables, GPSS generates the external table name using the job name.

Certificates:

When you specify gpfdist certificates, GPSS uses the SSL certificates and the gpfdists protocol to transfer encrypted data between itself and Greenplum Database.

CertFile: certfile_path: File system path to the server certificate.
KeyFile: keyfile_path: File system path to the server key file.
CAFile: CAfile_path: File system path to the Certificate Authority file. The CAfile_path must contain the entire Certificate Authority chain.
MinTLSVersion: min_version: The minimum transport layer security (TLS) version that GPSS requests on the connection. The default value is 1.0. GPSS supports minimum TLS versions of 1.0, 1.1, 1.2, and 1.3.
DBClientShared: bool_value: Determines whether GPSS shares this Gpfdist certificate. The default value is false, GPSS does not share the cert. When true, GPSS presents the Gpfdist certificate as the client certificate for the control channel connection to Greenplum Database. This configuration may be desirable if you use pgbouncer to manage connections to Greenplum Database.

BindAddress: bind_addr

The address from which GPSS listens for connections from Greenplum Database segments. Set bind_addr to an IP address that is reachable from every segment host by resolving the gpfdist_host configuration value, or set it to 0.0.0.0 to listen for connections from any host. The default bind address is 0.0.0.0.

Shadow Option

The encode/decode key for the Greenplum Database password.

Shadow: passwd_shadow_key: The key that GPSS uses to encode and decode shadow password strings. You can specify a shadowed password for the Greenplum Database user name. You can also specify a shadowed password for GPSS client to server Authentication. Keep this key secret.

Authentication Options

The user name and password that GPSS uses to authenticate the client program (gpsscli) with this GPSS server.

Username: client_auth_username: The user name with which the GPSS server instance authenticates a client.
SHADOW:shadowed_passwd_string: The shadowed password with which the GPSS server instance authenticates a client.

JobStore Option

The GPSS job information store.

File: Directory: jobstore_dir: The file system directory on the local host in which GPSS maintains current job information and status.

Logging Options

The back-end (log file) and front-end (command line output) logging levels for GPSS commands. GPSS supports the following log levels (from most to least severe): fatal, error, warn or warning, info, and debug.

BackendLevel: level: The logging level for messages that gpss, gpsscli, and gpkafka write to log files. The default back-end logging level is debug.
FrontendLevel: level: The logging level for messages that gpss, gpsscli, and gpkafka write to stdout. The default front-end logging level is info.
SplitByJob: level: Identifies if and how GPSS manages server log files. By default (no SpitByJob specified), GPSS creates a log file per server invocation. The only valid level is StartTime. When StartTime is specified, GPSS creates per-run server log files, and creates a new log file each time a job is started or loaded.; You may specify neither, or only one of SplitByJob or Rotate.
Rotate: policy_period: The time period that governs the GPSS server log file rotation policy. Valid policy period values are daily and hourly. By default (no Rotate specified), GPSS does not rotate a server log file on its own.; You may specify neither, or only one of SplitByJob or Rotate.

Monitor Option

The address to which GPSS binds the Prometheus scrape target for the GPSS server instance.

Prometheus: Listening: prom_addr:prom_port: The host name or IP address and port number from which the GPSS service instance allows Prometheus to pull the server's metrics.

KeepAlive Options

These properties control the gRPC connection between the GPSS client and GPSS server. Refer to the gRPC keepalive package documentation for more information about these properties and their default values.

MaxConnectionIdle: duration: The amount of time after which an idle connection is closed. The default value is infinity.
Time: duration: The amount of time of no activity after which a server pings the client to see if the transport is still alive. The default value is 2h.
Timeout: duration: The amount of time of no activity after a keepalive ping that prompts closure of the connection. The default value is 20s.
MinTime: duration: The mininum amount of time a client should wait before sending a keepalive ping. The default value is 5m.
PermitWithoutStream: bool_value: Determines if the server allows keepalive pings when there are no active streams. If false, the server will close the connection when a client sends a ping with no active streams. The default value is false.

Notes

When you provide the --config gpss.json option to the gpsscli shadow command, GPSS uses the Shadow:Key specified in the file to encode the password that you specify. If you do not provide a gpss.json file, GPSS uses a default key to encode the password. The gpsscli shadow command generates and outputs the encoded shadow password string.

When you provide a variant of this file to the other gpsscli subcommands via the --config gpsscliconfig.json option, GPSS uses the information provided in the ListenAddress block of the file to identify the GPSS server instance to which to route the request, and/or to identify the client certificates when SSL is enabled between GPSS client and server.

When you provide the --config gpfdistconfig.json option to the gpkafka load command, GPSS uses the information provided in the Gpfdist block of the file to specify the gpfdist protocol instance, and, when SSL is enabled on the data channel to Greenplum Database, to identify the GPSS SSL certificates.

Examples

Start a Greenplum Streaming Server instance with properties as defined in a configuration file named gpss4ic.json located in the current directory:

$ gpss --config gpss4ic.json

Example gpss4ic.json configuration file:

{
    "ListenAddress": {
        "Host": "",
        "Port": 5019
    },
    "Gpfdist": {
        "Host": "",
        "Port": 8319,
        "ReuseTables": false
    },
    "Shadow": {
        "Key": "a_very_secret_key"
    },
    "Monitor": {
        "Prometheus": {
            "Listening": "0.0.0.0:5001"
    },
    "JobStore": {
        "File": {
            "Directory": "/home/gpadmin/jobstore"
        }
    }
}

Synopsis

Description

Keywords and Values

Notes

Examples

See Also