Azure Data Factory: skip activity in debug mode

Viewed 44

Basic question
How can I skip an activity within a pipeline in Azure Data Factory if the pipeline runs in debug mode?

Background information
I have a complex pipeline setup (one master pipeline that triggers multiple sub pipelines) which also triggers fail messages if some activities failed. When testing things in debug mode, the fail messages are also triggered. This should not be happening to avoid spam.

Current approach
I could use the system variable @pipeline().TriggerType, which has the value Manual and pass that information as parameter from master pipeline through every single sub pipeline and check for the trigger type before sending the message (if triggerType != Manual). But this would mean a lot of changes and more things to consider when creating new pipelines, because that parameter always needs to be there then.

Does anyone have a better idea? Any idea how I can check in a sub-pipeline if the whole process was initially triggered via a scheduled trigger or as a debug run?

1 Answers

Currently we can't disable / skip an activity in ADF pipeline during its run

Please submit the feedback for this feature here:

https://feedback.azure.com/forums/270578-data-factory/suggestions/33111574-ability-to-disable-an-activity

You can either follow one of these for now:

  1. Manually delete the activity and click debug for execution but don't publish it

  2. Create a copy of that pipeline by cloning from original pipeline and delete the activities that you need to skip and save that with a suffix DEBUG which will become easy to identify and then you can run that pipeline whenever you need to debug

  3. Perform the steps using parameter as you mentioned

Thanks

Related