Table name is getting appending with column names in resultent file in azure datafactory

user_11027085 0
Viewed 125

I was trying to get data from On-prem hive Source to Azure data lake gen 2 using azure data factory. As I need to get data for multiple tables I have created and file(ex: tnames.txt) with all my table names and stored in data lake gen 2.

In Azure Data Factory created a lookup activity and passed tnames.txt file to it.

Then added a foreach activity to that lookup actvity and in foreach activity added a copy activity.

In copy activity in source, I was giving query to extract data. Sink is datalake gen 2.

Example code:

select * from tableName

Here table is dynamically passed from tnames.txt.

But after data is copied into data lak,e I am getting headers in copied data are like:

 "tablename.columnname".

For example: Table name is Employee and few columns are ID, Name, Gender,....

My resultent file columns are like Employee.ID,Employee.Name,Employee.Gender, but my requirement is just column name.

Basically tabe name is append to column name.

How to solve this issue/Is there any other way to get data for multiple tables in single pipeline/copy activity?

azure-data-factory-2
1 Answers

Check the mapping tab of your copy activity . If the mapping is enabled, clear it and use auto-create table . It will auto-generate the schema according to the source schema. No need to explicitly create the table with defined schema. Let it be auto create table. It will generate required mapping automatically.

user_177211240
Related