You can use the VMware Tanzu Greenplum Connector for Apache NiFi in an Apache NiFi dataflow to load record-oriented data from any source into VMware Tanzu Greenplum. The Connector uses the VMware Tanzu Greenplum Streaming Server to load the data in parallel.
The following table identifies the supported component versions for the VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.x:
|Connector Version||Apache NiFi Version||Greenplum Version||Greenplum Streaming Server Version|
|1.1.0, 1.0.1||1.10.x, 1.11.x, 1.12.x||6.x||1.4.1+1|
|1.0.0||1.10.x, 1.11.x, 1.12.x||6.x||1.4.1 - 1.6.0|
1 Refer to Known Issues and Limitations for specific version caveats.
Component documentation references:
If you are currently using the VMware Tanzu Greenplum Connector for Apache NiFi, you may be required to perform upgrade actions for this release. Review Upgrading the Connector to plan your upgrade.
Release Date: September 9, 2022
When a column in the target Greenplum Database table has no matching field in an incoming NiFi record and the
Unmatched Column Behavior configuration setting specifies either
Ignore Unmatched Columns or
Warn on Unmatched Columns, the VMware Tanzu Greenplum Connector for Apache NiFi sends no column value for the record to GPSS. If the Greenplum table definition includes a default value for the column, Greenplum Database writes the tuple with the default column value; otherwise Greenplum writes the tuple with a NULL column value.
VMware Tanzu Greenplum Connector for Apache NiFi 1.1.0 resolves these issues:
Release Date: June 14, 2022
VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.1 adds support for Greenplum Streaming Server versions 1.6.1+.
Release Date: December 16, 2020
The VMware Tanzu Greenplum Connector for Apache NiFi 1.0.0 release supports inserting, merging, and updating record-oriented data in Greenplum Database. You can set up an Apache NiFi dataflow that sends Avro, CSV, Parquet, JSON, or XML data to the Connector
PutGreenplumRecord processor to write to a Greenplum table.
The VMware Tanzu Greenplum Connector for Apache NiFi 1.0.x has these known issues and limitations:
timestamp without timezonetype data into Greenplum Database may expose a data type conversion issue. This issue is resolved in Greenplum Streaming Server version 1.7.1; be sure to use the VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.1+ should you encounter this scenario.
"ReuseTables": falsein the
gpss.jsonserver configuration file that you use to start the Greenplum Streaming Server.
QueryDatabaseTableRecordprocessor to the
PutGreenplumRecordprocessor must set the
Use Avro Logical Typesproperty to
trueto send date and time data in the format supported by the Connector.
ParquetReaderprocessor to fail when you use it with Apache NiFi version 1.12.