GPSS supports authenticating with Kerberos to obtain both Kafka and Greenplum Database credentials. GPSS supports authenticating with LDAP to obtain Kafka credentials. GPSS also supports using TLS/SSL to encrypt communication between Kafka and GPSS, between RabbitMQ and GPSS, between the GPSS client and server, and on the data and control channels between GPSS/gpkafka and Greenplum.

Note

GPSS does not support Kerberos authentication to RabbitMQ.

Configuring gpss and gpkafka for TLS-Encrypted Communications with Kafka

If your Kafka version 0.9 and newer cluster is configured to use TLS encryption, you must configure GPSS to use this encryption method when communicating with Kafka. You perform this configuration at both the GPSS service instance and client levels.

Refer to the Apache Kafka Encryption and Authetication using SSL documentation for more information about SSL configuration.

  1. Create Kafka client keys for the gpss or gpkafka instance.

  2. Specify the location of the GPSS client certificates via Kafka properties in the PROPERTIES or rdkafka_prop (version 3 (Beta)) block of the load configuration file. For example:

    PROPERTIES:
    security.protocol: SSL
    ssl.ca.location: /path/to/cert/kafka-ca.crt
    ssl.certificate.location: /path/to/cert/gpssclient.crt
    ssl.key.location: /path/to/cert/gpssclient.key

  3. If you are using the gpsscli subcommands to load data, ensure that the ListenAddress:Host that you specify for the GPSS server identifies the common name (CN) in the certificate.

Configuring gpss for TLS-Encrypted Communications with RabbitMQ

Note

GPSS supports TLS only when loading from a RabbitMQ queue. GPSS does not support TLS when loading from a RabbitMQ stream.

If your RabbitMQ cluster is configured to use TLS encryption, you may configure GPSS to use this encryption method when it communicates with RabbitMQ. You perform this configuration at both the GPSS service instance and client levels.

Refer to the RabbitMQ TLS Support documentation for more information about TLS configuration.

  1. Create RabbitMQ client keys for the gpss instance. Refer to the TLS Certificate Generator github repository for information about generating RabbitMQ certificates.

  2. Specify the RabbitMQ server in the load configuration file SERVER or server (version 3 (Beta)) property; you must omit the user name and password. For example (version 2):

    SERVER: localhost:5672

  3. Specify the location of the GPSS client certificates via RabbitMQ properties in the PROPERTIES or properties (version 3 (Beta)) block of the load configuration file. For example:

    PROPERTIES:
    use.ssl: true
    ssl_options.servername: "hostname"
    ssl_options.cacertfile: /path/to/cert/rabbit/ca/cacert.pem
    ssl_options.certfile: /path/to/cert/rabbit/client/rabbit-client.cert.pem
    ssl_options.keyfile: /path/to/cert/rabbit/client/rabbit-client.key.pem

  4. Optional RabbitMQ TLS configuration properties and example settings include the following:

    ssl_options.verify:  verify_peer
ssl_options.fail_if_no_peer_cert: true

  5. If you are using the gpsscli subcommands to load data, ensure that the ListenAddress:Host that you specify for the GPSS server identifies the common name (CN) in the certificate.

Configuring gpss and gpkafka for SSL-Encrypted Communications with Greenplum

There are two communication channels between GPSS and Greenplum Database: a control channel and a data channel. GPSS supports SSL encryption on both channels.

Configuring SSL for the Data Channel

GPSS supports SSL encryption on the data channel to Greenplum using the gpfdists protocol for encrypted communications.

If your Greenplum Database cluster is configured to use SSL, you must configure GPSS to use this encryption method for the data channel when it communicates with Greenplum.

  1. Create GPSS keys for the gpfdist protocol instance.

  2. Configure the gpfdist protocol to use SSL encryption to Greenplum by providing a Gpfdist:Certificate block in the GPSS configuration file, and identify the file system location of the SSL certificates. You can also specify a minimum TLS version. Sample gpss.json or gpfdistconfig.json excerpt:

    "Gpfdist": {
    "Host": "127.0.0.1",
    "Port": 5001,
    "Certificate": {
        "CertFile": "/home/gpadmin/cert/gpss.crt",
        "KeyFile": "/home/gpadmin/cert/gpss.key",
        "CAFile": "/home/gpadmin/cert/root_client.crt"
        "MinTLSVersion": "1.2"
    }
}

  3. If you are using gpkafka to load data, ensure that the Gpfdist:Host that you specify identifies the common name (CN) in the certificate.

Configuring SSL for the Control Channel

GPSS supports SSL encryption on the control channel to Greenplum as described in the PostgreSQL SSL Support documentation. GPSS uses SSL only for encryption on the control channel; it does not check the certificates. The default SSL mode that GPSS uses on this channel is the prefer mode.

You have two options for configuring GPSS client SSL for the control channel:

  1. Option 1: Instruct GPSS to re-use the data channel (Gpfdist) certificate. Configuration steps include:

    1. Locating the Gpfdist:Certificate block in the gpss.json GPSS server configuration file.

    2. Setting the DBClientShared property to true:

      "Gpfdist": {
    ...
    "Certificate": {
        ...
        "DBClientShared": true
    }
}

  2. Option 2: Locate the certificates and keys in the ~/.postgresql directory on the GPSS server host. Configuration steps include:

    1. Copying the root certificate to the ~/.postgresql directory on the GPSS server host.
    2. Generating a private key and client certificate for the GPSS server and copying both to the ~/.postgresql directory on the GPSS server host.

Configuring gpss and gpsscli for Encrypted gRPC Communications

GPSS supports encrypting communications between the gpsscli client and gpss server.

To use encrypted gRPC on connections between gpsscli and gpss, you must create server and client keys, and provide the keys via configuration files that you provide to the commands.

  1. Create server keys for the gpss server instance.

  2. Create client keys for the gpsscli client.

  3. Configure the gpss service instance to use SSL encryption to the client by providing a ListenAddress:Certificate block in the gpss.json GPSS configuration file. The properties in this block should identify the file system location of the SSL server keys. You can also specify the minimum TLS version. Sample gpss.json excerpt:

    "ListenAddress": {
    "Host": "",
    "Port": 5019,
    "Certificate": {
        "CertFile": "/home/gpadmin/cert/gpss.crt",
        "KeyFile": "/home/gpadmin/cert/gpss.key",
        "CAFile": "/home/gpadmin/cert/root_cli.crt"
        "MinTLSVersion": "1.2"
    }
}

  4. Configure the GPSS client to use SSL encryption to the server by specifying the client keys in the ListenAddress:Certificate block of a GPSS configuration file that you provide to the gpsscli subcommand via the --config gpsscliconfig.json option. Sample gpsscliconfig.json excerpt:

    "ListenAddress": {
    "Host": "",
    "Port": 5019,
    "Certificate": {
        "CertFile": "/home/gpadmin/cert/gpsscli.crt",
        "KeyFile": "/home/gpadmin/cert/gpsscli.key",
        "CAFile": "/home/gpadmin/cert/root.crt"
    }
}

If you encrypt communications between the GPSS client and server, but you want to deactivate certificate verification, specify the --no-check-ca option when you run the gpsscli subcommand.

Configuring gpss and gpkafka for Kerberos Authentication to Greenplum

If Kerberos authentication is enabled for Greenplum Database, you must configure gpss to authenticate with Kerberos.

GPSS uses a kerberos ticket, and the USER name specified in the load configuration file, to connect to Greenplum Database.

  1. Create a Kerberos principal for each Greenplum Database user that will use GPSS to load data into Greenplum.
  2. Specify the principal name in the load configuration file USER property value.
  3. Generate a Kerberos ticket for this principal before you submit a load job with the gpsscli submit, gpsscli load, or gpkafka load commands.
Note

If your Greenplum Database Kerberos service name is not the default (postgres), set the PGKRBSRVNAME environment variable to the correct service name before you start the gpss service instance or run gpkafka load.

Configuring gpss for Kerberos Authentication to Kafka

If your Kafka version 0.9 and newer cluster is configured for Kerberos authentication, you must configure GPSS to use this authentication method. You perform this configuration at both the gpss service instance level and the GPSS client level.

GPSS is a Kafka client. You must create a Kerberos principal for the gpss server instance accessing Kafka, and generate a keytab file for this principal. By default, GPSS runs kinit using this principal and keytab to generate the Kerberos ticket.

You must set certain Kafka properties in your load configuration file to use Kerberos user authentication to Kafka. The following table identifies keywords and values that you can add to the PROPERTIES or rdkafka_prop (version 3 (Beta)) block in your load configuration file:

Keyword Value
security.protocol The Kafka security protocol. Obtain the value from the Kafka server server.properties configuration file. GPSS supports the SASL_SSL (Kerberos and SSL) and SASL_PLAINTEXT (Kerberos, no SSL) protocols.
sasl.kerberos.keytab The absolute path to the GPSS or user Kerberos keytab file for Kafka on the local system.
sasl.kerberos.kinit.cmd The Kerberos kinit command string. If this property is not specified, GPSS uses the default value as described in librdkafka Global configuration properties when it runs the kinit command. If you do not want GPSS to run kinit, set the sasl.kerberos.kinit.cmd property to an empty value ("") or no value.
sasl.kerberos.principal The GPSS or user Kerberos service principal name; typically of the format <name>@<realm> or <primary>/<instance>@<realm>.
sasl.kerberos.service.name The Kafka Kerberos principal name. Obtain the value from the Kafka server server.properties configuration file. The default Kafka Kerberos service name is kafka.

For example:

PROPERTIES:
    security.protocol: SASL_PLAINTEXT
    sasl.kerberos.service.name: kafka
    sasl.kerberos.keytab: /var/kerberos/krb5kdc/gpss.keytab
    sasl.kerberos.principal: gpss/[email protected]
    sasl.kerberos.kinit.cmd:

If you are accessing Kafka using both Kerberos authentication and SSL encryption, you must also specify the Kafka SSL properties identified in Configuring gpss and gpkafka for SSL-Encrypted Communications with Kafka.

Configuring gpss for LDAP Authentication to Kafka

If your Kafka cluster is configured for LDAP authentication, you must configure GPSS to use this authentication method. Set Kafka properties in your load configuration file as follows:

  1. Edit the BROKERS to specify a broken that supports LDAP authentication.

  2. Add keywords and values to the PROPERTIES or rdkafka_prop (version 3 (Beta)) block in your load configuration file as needed:

    Keyword Value
    security.protocol Specify SASL_PLAINTEXT as the Kafka security protocol when authenticating to LDAP.
    sasl.mechanism Specify PLAIN for the security mechanism.
    sasl.username Enter the LDAP user name.
    sasl.password Enter the LDAP user password.

    GPSS also supports using a shadow string for the LDAP user password with the gpsscli shadow command. If you generate a shadowed password, specify the password using the format:

    sasl.password: "SHADOW:<shadow_password_string>"
check-circle-line exclamation-circle-line close-line
Scroll to top icon