We are able to load uncompressed CSV files and gzipped files completely fine. However, if we want to load CSV files compressed in ".zip" - what is the best approach to move ahead? Will we need to manually convert the zip to gz or BigQuery has added some support to handle this?
Thanks
BigQuery supports loading gzip files
The limitation is - If you use gzip compression BigQuery cannot read the data in parallel. Loading compressed CSV data into BigQuery is slower than loading uncompressed data.
You can try 42Layers.io for this. We use it to import ziped CSV files directly from FTP into BQ, and then set it on a schedule to do it every day. They also let you do field mapping to your existing tables within BQ. Pretty neat.
