You can use the VMware Greenplum Connector for Apache NiFi in an Apache NiFi dataflow to load record-oriented data from any source into VMware Greenplum. The Connector uses the VMware Greenplum Streaming Server to load the data in parallel.
The following table identifies the supported component versions for the VMware Greenplum Connector for Apache NiFi version 1.0.x:
Connector Version | Apache NiFi Version | Greenplum Version | Greenplum Streaming Server Version |
---|---|---|---|
1.1.0, 1.0.1 | 1.10.x, 1.11.x, 1.12.x | 6.x | 1.4.1+1 |
1.0.0 | 1.10.x, 1.11.x, 1.12.x | 6.x | 1.4.1 - 1.6.0 |
1 Refer to Known Issues and Limitations for specific version caveats.
Component documentation references:
If you are currently using the VMware Greenplum Connector for Apache NiFi, you may be required to perform upgrade actions for this release. Review Upgrading the Connector to plan your upgrade.
Release Date: December 20, 2023
VMware Greenplum Connector for Apache NiFi 1.1.1 resolves these issues:
commons-text
library.
Release Date: September 9, 2022
When a column in the target Greenplum Database table has no matching field in an incoming NiFi record and the PutGreenplumRecord
processor Unmatched Column Behavior
configuration setting specifies either Ignore Unmatched Columns
or Warn on Unmatched Columns
, the VMware Greenplum Connector for Apache NiFi sends no column value for the record to GPSS. If the Greenplum table definition includes a default value for the column, Greenplum Database writes the tuple with the default column value; otherwise Greenplum writes the tuple with a NULL column value.
VMware Greenplum Connector for Apache NiFi 1.1.0 resolves these issues:
Release Date: June 14, 2022
VMware Greenplum Connector for Apache NiFi version 1.0.1 adds support for Greenplum Streaming Server versions 1.6.1+.
Release Date: December 16, 2020
The VMware Greenplum Connector for Apache NiFi 1.0.0 release supports inserting, merging, and updating record-oriented data in Greenplum Database. You can set up an Apache NiFi dataflow that sends Avro, CSV, Parquet, JSON, or XML data to the Connector PutGreenplumRecord
processor to write to a Greenplum table.
The VMware Greenplum Connector for Apache NiFi 1.0.x has these known issues and limitations:
timestamp without timezone
type data into Greenplum Database may expose a data type conversion issue. This issue is resolved in Greenplum Streaming Server version 1.7.1; be sure to use the VMware Greenplum Connector for Apache NiFi version 1.0.1+ should you encounter this scenario."ReuseTables": false
in the gpss.json
server configuration file that you use to start the Greenplum Streaming Server.QueryDatabaseTableRecord
processor to the PutGreenplumRecord
processor must set the Use Avro Logical Types
property to true
to send date and time data in the format supported by the Connector.ParquetReader
processor to fail when you use it with Apache NiFi version 1.12.