Every time I submit and execute a Run in Azure Machine Learning, I end up getting logs. I can see the logs by going to the Run page and clicking on 'Outputs+logs' tab (see the image). Here I am curious about where the logs actually are. I have an Azure Blob storage but can't find the logs there. Where are they?
2 Answers
Every Azure Machine Learning workspace creates automatically 4 other Azure resources under the same resource group:
- Azure Storage account: Is used as the default datastore for the workspace - where the logs are stored.
- Azure Container Registry: Registers docker containers that you use during training and when you deploy a model.
- Azure Application Insights: Stores monitoring information about your models.
- Azure Key Vault: Stores secrets that are used by compute targets and other sensitive information that's needed by the workspace.
You can see this resources under your resource group, with the same name as your AML workspace + random numbers. If you select the storage account, select the option Containers in the left hand side menu. In the main pane, select azureml > ExperimentRun, and all the runIDs will be represented by an individual folder, where you can find the logs.
Alternatively, you can use the Azure Portal UI to go to the AML Workspace > Experiments > Select the Experiment > In the Run list, select the run you want. On the details tab, the last option is See all properties (Raw JSON). From there you can see both the RunId and the full path of the logfiles.
If you want more information about what type of log is stored in which file, check here.
In the storage account access to the container azureml, path:
azureml / ExperimentRun / dcid.XXX / logs / azureml
Where XXX is the run id of the step.
And there you can find the execution logs, stderr and stdout files.
Related