Trigger Jupyter Notebook in Azure ML workspace from ADF

user_10382230 0
Viewed 1244

How do I trigger a notebook in my Azure Machine Learning notebook workspace from Azure Data Factory

I want to run a notebook in my Azure ML workspace when there are changes to my Azure storage account.

azure-machine-learning-service azure-data-factory-2 azure-data-factory
2 Answers

My understanding is that your use case is 100% valid and it is currently possible with the azureml-sdk. It requires that you create the following:

  1. Create an Azure ML Pipeline. Here's a great introduction.
  2. Add a NotebookRunnerStep to your pipeline. Here is a notebook demoing the feature. I'm not confident that this feature is still being maintained/supported, but IMHO it's a valid and valuable feature. I've opened this issue to learn more
  3. Create a trigger using Logic apps to run your pipeline anytime a change in the datastore is detected.

There's certainly a learning curve to Azure ML Pipelines, but I'd argue the payoff is in the flexibility you get in composing steps together and easily scheduling and orchestrating the result.

user_38426100

This feature is currently supported by Azure ML Notebooks. You can also use Logic apps to trigger a run of your Machine Learning pipeline when there are changes to your Azure storage account.

user_119688550
Related