Azure Data Factory V2 - Compression type (enum) as parameter

user_297606 0
Viewed 153

I have a pipeline to copy blobs between storages.

I'd like to have Compression Type and Compression Level in the Sink Dataset (storage) set up as parameters.

When I edit the dataset as JSON, I'm able do use expressions and parameters, but there's no way to switch compression off (None), which would be represented as an empty object.

Is there a way to achieve this?

Thanks!

azure-data-factory-2
1 Answers

Ok, I finally figured it out - you have to directly edit JSON for the target linked storage service, and the expression I used is

            "compression": {
            "type": "Expression",
            "value": "@json(if(or(equals(dataset().TargetCompressionType, 'None'),equals(dataset().TargetCompressionType, '')),'{}',concat('{\"type\":\"', dataset().TargetCompressionType, '\", \"level\":\"', dataset().TargetCompressionLevel, '\"}')))"
        }

So when TargetCompressionType is set to empty string or "None", the result will be an empty object and compression won't be used, and when set to "ZipDeflate" or others, the files from source will be compressed.

user_2976060
Related