How to automatically start and stop a compute instance to execute Azure Machine Learning pipelines

user_18048190 0
Viewed 820

I've been following this notebook to schedule the execution of some code every hour. It makes a schedule and creates the pipelines. The problem is that if the computer instance is not running, the pipeline is just queued and waiting for it to run.

Is there a way to automatically run a compute instance when a pipeline is triggered within Azure Machine Learning, as well as stop them when the pipeline is completed?

I need to do it within the AzureML Studio platform because that is the only thing external Data Scientists have access to. I can't use clusters because some of their behaviors cause issues with the code.

I can schedule an instance to become active approximately at the same time as the schedule, but I want to do it in the code, so the scripts can run when they are up, as well as shut down the instance when the pipeline run is over.

azure-machine-learning-studio azure-machine-learning-service azure-devops azure
1 Answers

There are several ways to manage the compute instance, you can set it up either in studio or in script.

  1. in studio: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-manage-compute-instance?tabs=python#create-a-schedule-in-studio

  2. with Resource Manage Template: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-manage-compute-instance?tabs=python#create-a-schedule-with-a-resource-manager-template

  3. in script: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-manage-compute-instance?tabs=python#setup-script

user_95988010
Related