Does Microsoft has any documentation. I need complete information to a run pipeline, i.e
Start time, end time, pipeline job id, no of record inserted, deleted, update, error, etc
3 Answers
If you use Python, you can use Azure Python SDK to access the pipeline_runs and activity_runs metadata and write to a database table.
Currently, Microsoft allows you to pull all the pipeline execution information using the REST api : It's a simple GET Request : https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}?api-version=2018-06-01
Check out this link for more information on the API : https://docs.microsoft.com/en-us/rest/api/datafactory/pipelineruns/get
Related