Difference between datatypes supported by Glue, Athena and Spark

Viewed 1680

I have a datalake constructed on top of AWS S3. I'm using Glue catalog for storing the metadata of datalake tables. These tables will be queried using Athena and spark for various purpose.

While defining the table columns, I noticed that the data types supported by Glue, Spark and Athena are not same. Below links shows the datatypes supported by Glue, Athena and Spark

glue : https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-common.html

athena : https://docs.aws.amazon.com/athena/latest/ug/data-types.html

spark : https://spark.apache.org/docs/latest/sql-ref-datatypes.html

Keeping performance in mind, which datatypes should I use while creating datalake tables,

0 Answers
Related