When you load Greenplum data into Spark, or write Spark data into Greenplum, the VMware Greenplum Connector for Apache Spark maps Greenplum Database and Spark data types for you.

Note: The Connector does not support complex types, nor does it support any data type not listed in the tables below.

For additional information on Greenplum Database or Spark data types, refer to:

Greenplum to Spark

The mapping from Greenplum Database data type to Spark data type (read) is identified in the table below:

Greenplum Data Type Spark Data Type
bigint LongType
bigSerial LongType
boolean BooleanType
char StringType
date DateType
decimal DecimalType
float4 FloatType
float8 DoubleType
int IntegerType
numeric DecimalType
serial IntegerType
smallInt ShortType
text StringType
time TimeStampType
timestamp TimeStampType
timestamptz TimeStampType
timetz TimeStampType
varchar StringType

Spark to Greenplum

The mapping from Spark data type to Greenplum Database data type (write) is identified in the table below:

Spark Data Type Greenplum Data Type
BinaryType bytea
BooleanType boolean
DateType date
DecimalType numeric
DoubleType float8
FloatType float4
IntegerType int
LongType bigint
ShortType smallInt
StringType text
TimeStampType timestamp
check-circle-line exclamation-circle-line close-line
Scroll to top icon