You can use the VMware Tanzu Greenplum Connector for Apache NiFi in an Apache NiFi dataflow to load record-oriented data from any source into VMware Tanzu Greenplum. The Connector uses the VMware Tanzu Greenplum Streaming Server to load the data in parallel.

Supported Platforms

The following table identifies the supported component versions for the VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.x:

Connector Version Apache NiFi Version                 Greenplum Version Greenplum Streaming Server Version
1.1.0, 1.0.1 1.10.x, 1.11.x, 1.12.x 6.x 1.4.1+1
1.0.0 1.10.x, 1.11.x, 1.12.x 6.x 1.4.1 - 1.6.0

1 Refer to Known Issues and Limitations for specific version caveats.

Component documentation references:

Upgrading to Version 1.x

If you are currently using the VMware Tanzu Greenplum Connector for Apache NiFi, you may be required to perform upgrade actions for this release. Review Upgrading the Connector to plan your upgrade.

Release 1.1.0

Release Date: September 9, 2022

Changes

When a column in the target Greenplum Database table has no matching field in an incoming NiFi record and the PutGreenplumRecord processor Unmatched Column Behavior configuration setting specifies either Ignore Unmatched Columns or Warn on Unmatched Columns, the VMware Tanzu Greenplum Connector for Apache NiFi sends no column value for the record to GPSS. If the Greenplum table definition includes a default value for the column, Greenplum Database writes the tuple with the default column value; otherwise Greenplum writes the tuple with a NULL column value.

Resolved Issues

VMware Tanzu Greenplum Connector for Apache NiFi 1.1.0 resolves these issues:

32345
Resolves an issue where a default column value was not applied when a NiFi record had no matching field; see Changes above.

Release 1.0.1

Release Date: June 14, 2022

Changes

VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.1 adds support for Greenplum Streaming Server versions 1.6.1+.

Release 1.0.0

Release Date: December 16, 2020

Features

The VMware Tanzu Greenplum Connector for Apache NiFi 1.0.0 release supports inserting, merging, and updating record-oriented data in Greenplum Database. You can set up an Apache NiFi dataflow that sends Avro, CSV, Parquet, JSON, or XML data to the Connector PutGreenplumRecord processor to write to a Greenplum table.

Known Issues and Limitations

The VMware Tanzu Greenplum Connector for Apache NiFi 1.0.x has these known issues and limitations:

  • Using the Connector with Greenplum Streaming Server versions prior to v1.7.1 when loading timestamp without timezone type data into Greenplum Database may expose a data type conversion issue. This issue is resolved in Greenplum Streaming Server version 1.7.1; be sure to use the VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.1+ should you encounter this scenario.
  • If you run the Connector against Greenplum Streaming Server version 1.4.x or older, you must specify "ReuseTables": false in the gpss.json server configuration file that you use to start the Greenplum Streaming Server.
  • The Connector does not support SSL-encrypted connections to the Greenplum Streaming Server.
  • A dataflow that you create that routes Avro data from the QueryDatabaseTableRecord processor to the PutGreenplumRecord processor must set the Use Avro Logical Types property to true to send date and time data in the format supported by the Connector.
  • Apache NiFi bug NIFI-7817 causes the built-in ParquetReader processor to fail when you use it with Apache NiFi version 1.12.
  • Decimal types may lose precision when you use the Connector with Apache NiFi versions 1.10 and 1.11.
check-circle-line exclamation-circle-line close-line
Scroll to top icon