AWS S3 - Athena without gz compression

user_3605831 0
Viewed 2102

I'm creating a table with the following statement in AWS Athena:

CREATE TABLE table2
WITH 
(
  format='JSON',
  external_location='s3://bucket/path'
) AS
SELECT field1, field2, field3, field4, field5, field6
FROM table1
ORDER BY field1, field2

However, when I look into external_location, I see that all the files are compressed into gz format. Is it possible to turn off compression? That is, just write raw JSON to the S3 bucket.

amazon-athena amazon-s3
2 Answers

It is currently not possible to create uncompressed files with Athena's CTAS feature. For ORC and Parquet you can choose the compression type, but for all other formats gzip will be used whether you like it or not.

It's a bit ironic that while you can't get CTAS output uncompressed, there is no way to get regular query output compressed.

user_11090

Since ~May 2022 (I didn't see any announcements about this...), you are able to disable the compression. You need to specify the write_compression in the TBL_PROPERTIES / WITH part of the query:

CREATE TABLE new_table
WITH (write_compression = 'NONE', format = 'TEXTFILE')
AS SELECT ...

All compressions (including 'NONE') per formats are availables here: https://docs.aws.amazon.com/athena/latest/ug/compression-formats.html

user_41801780
Related