How to create Event Trigger in Azure Data Factory when three files created in Azure Blob Container?

user_6391507 0
Viewed 346

I need to create a schedule trigger (it will run every 15 minutes for 3 hours) in Azure Data Factory, which will pipeline when three different files are created in an Azure Blob storage container. Pipeline execution should only start when all 3 files are created in the blob container. For example, if 3 hours pass and there are only two files in the storage blob, the pipeline will not have to run.

azure-triggers azure-data-factory-2 azure-data-factory azure-blob-storage azure
1 Answers

There is no direct way for the event trigger of 3 files as an AND condition in ADF as of now. What you can do is:

  1. Create an ADF pipeline with a) Get meta data activity>> check whether there are 3 required files b) If yes, then use Execute pipeline activity to trigger the pipeline that should be run when there are 3 files If not, ignore/throw error etc

Create Event triggers for the files and associate with the pipeline.

So in the case of 3rd event trigger, all files would be found and then the main pipeline would be executed.

user_114054230
Related