When you load Greenplum data into Spark, or write Spark data into Greenplum, the maps Greenplum Database and Spark data types for you.
Note: The Connector does not support complex types, nor does it support any data type not listed in the tables below.
For additional information on Greenplum Database or Spark data types, refer to:
Greenplum to Spark
The mapping from Greenplum Database data type to Spark data type (read) is identified in the table below:
| Greenplum Data Type | Spark Data Type |
|---|---|
| bigint | LongType |
| bigSerial | LongType |
| boolean | BooleanType |
| char | StringType |
| date | DateType |
| decimal | DecimalType |
| float4 | FloatType |
| float8 | DoubleType |
| int | IntegerType |
| numeric | DecimalType |
| serial | IntegerType |
| smallInt | ShortType |
| text | StringType |
| time | TimeStampType |
| timestamp | TimeStampType |
| timestamptz | TimeStampType |
| timetz | TimeStampType |
| varchar | StringType |
Spark to Greenplum
The mapping from Spark data type to Greenplum Database data type (write) is identified in the table below:
| Spark Data Type | Greenplum Data Type |
|---|---|
| BinaryType | bytea |
| BooleanType | boolean |
| DateType | date |
| DecimalType | numeric |
| DoubleType | float8 |
| FloatType | float4 |
| IntegerType | int |
| LongType | bigint |
| ShortType | smallInt |
| StringType | text |
| TimeStampType | timestamp |
