In Azure data factory, is it possible to control the filenames of numerous output files without using a data flow?

user_13690713 0
Viewed 286

For instance, I may use a copy activity in data factory to copy a 10 million record customer table into an Azure data lake, and use partition option of 'dynamic range' in the source options. My understanding is that this would result in data factory splitting the data into numerous files in the lake.

Using this method, how do I force a naming convention for the outputted files in the lake? e.g. so each of the filenames begin with 'cust_', meaning the files would be called cust_1, cust_2, cust_3, cust_4 etc.

azure-data-factory-pipeline azure-data-factory-2 azure-data-factory
1 Answers

My understanding is partition option of 'dynamic range' is used to split a file into multiple files and then perform asynchronous copying. This is a multi-threaded operation to increase the copy speed. It is Copy activity performance optimization features. I think this is not the file splitting you want.

  1. Select 'None' at source setting. enter image description here

  2. We can set 'File extension', 'Max rows per file' and 'File name prefix' at sink setting. enter image description here

  3. In my side, ADF will automatically split into multiple files, each file contains 50 rows of records. enter image description here

user_139794870
Related