Can someone suggest which of below option will be more performant with PUT command:
- Uploading file with AUTO_COMPRESS=true.
- Uploading compressed file(gzip) AUTO_COMPRESS=false.
Can someone suggest which of below option will be more performant with PUT command:
There's no harm to leaving AUTO_COMPRESS=true because if a file is already compressed, the PUT command won't try to double compress it. There is an important caveat to note though. If a file is already compressed, it must be compressed in a supported compression method. You can get a list of supported methods here: https://docs.snowflake.com/en/sql-reference/sql/put.html
Using compression either before or auto_compress is advisable since it will reduce network transfer times and bandwidth consumption. This will use CPU and IO on the server doing the PUT operation. If the server doing the PUT is maxed out (I've seen some cases of VMs on oversubscribed systems for example), it would be better to perform the compression before sending to the machine doing the PUT. This is because there's already a lot of CPU and IO on the PUT operation because it's encrypting the files prior to upload.