As a RabbitMQ consumer, GPSS manages the progress of each load operation using RabbitMQ's broker-based offset management. This management involves identifying how, when (before commit, after commit, or never), and where (history table, broker, both, nowhere) the offset is stored, and is directed by a combination of RabbitMQ client and GPSS load configuration properties.

RabbitMQ Properties

When loading from a RabbitMQ queue or stream, GPSS uses a default of gpss_rabbitmq_consumer_<job_id>. You can set the consumer name by specifying this RabbitMQ client property in the PROPERTIES (version 2) or properties (version 3 (beta)) load configuration file block. For example:

PROPERTIES: my_rmq_consumer

When loading from a RabbitMQ queue, you may also choose to set the consumer.exclusive and qos.prefetch.count client properties, which specify the number of queue consumers and prefetch limits per consumer. GPSS sets the consumer.exclusive property to false by default.

GPSS Properties

GPSS uses the CONSISTENCY (version 2) or consistency (version 3 (Beta)) load configuration file property setting to govern how it manages offsets.

GPSS supports the following CONSISTENCY settings for RabbitMQ:

CONSISTENCY: { strong | at-least | at-most | none }

GPSS supports strong consistency for RabbitMQ streams only.


The following table summarizes the offset commit behaviour for load jobs for a GPSS RabbitMQ consumer:

Consistency Value Behaviour
strong (available for RabbitMQ streams only) GPSS stores offsets in a history table.
at-least [or empty] GPSS stores in the offset in the RabbitMQ broker before commit. Note that this could result in duplicate data written to Greenplum Database when the connection to RabbitMQ closes after GPSS writes, but before it sends the response.
at-most GPSS stores the offset in the RabbitMQ broker after commit.
none GPSS does not store offsets anywhere.
check-circle-line exclamation-circle-line close-line
Scroll to top icon