Remove specific columns using Azure Data Factory

user_11049287 0
Viewed 3210

I have set of files in ADLS. Number of columns varies in each file.

File 1 will have the columns : Row_Number, Col_A, Col_B, null

File 2 will have the columns : Row_Number, Col_1, Col_2, Col_3 , null

File 3 will have the columns : Row_Number, Col_A1, Col_A2, null

There will be more than 50 files in my ADLS. I would need to remove first and last column in each file or to be specific , I need to remove columns which has columns names as Row_Number and null.

Can any help me what should be my expressions in Derived column / Select.

Thanks for your help.

azure-data-factory azure
1 Answers

I would use a Select rule with this for the matching condition:

name != 'Row_Number' && name != 'null' && left(name,2) != '_c'

and this for the output column name: $$

enter image description here

user_73507880
Related