What is the difference between ADF Pipeline and ADF Data flow? Why are sinks/sources supported in Pipeline and Data flow different? Is it possible to create a pipeline that reads data from source, filter, use joins and store data to a sink without a data flow? Please let me know.
1 Answers
Pipelines are for process orchestration. Data Flow is for data transformation.
In ADF, Data Flows are built on Spark using data that is in Azure (blob, adls, SQL, synapse, cosmosdb).
Connectors in pipelines are for copying data and job orchestration. There are 90+ connectors available there that stretch across on-prem and other clouds.
We are always incrementally adding more connectors to data flows.
Related