AzureML webservice deployment with custom Environment - /var/runit does not exist

Question

I'm struggling to deploy a model with a custom environment through the azureml SDK.

I have built a docker image locally and pushed it to azure container registry to use it for environment instantiating. This is how my dockerfile looks like:

FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04
FROM python:3.9.12
        
# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
        
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1
        
# Install requirement for deploying the service
RUN apt-get update
RUN apt-get install -y runit
        
# Install pip requirements
RUN pip install --upgrade pip
COPY requirements.txt .
RUN pip install azureml-defaults
RUN pip install -r requirements.txt

I want to deploy the webservice locally for testing, so I am following the steps according to official documentation:

ws = Workspace(
    subscription_id='mysub_id', 
    resource_group='myresource_group', 
    workspace_name='myworkspace'
)
        
model = Model.register(
    ws, 
    model_name='mymodel', 
    model_path='./Azure_Deployment/mymodel_path'
)
        
container = ContainerRegistry()
container.address = 'myaddress'
myenv = Environment.from_docker_image('myenv_name', 'img/img_name:v1', container)
        
inference_config = InferenceConfig(
    environment=myenv, 
    source_directory='./Azure_Deployment', 
    entry_script='echo_score.py',
)
        
deployment_config = LocalWebservice.deploy_configuration(port=6789)
        
service = Model.deploy(
    ws, 
    "myservice", 
    [model], 
    inference_config, 
    deployment_config, 
    overwrite=True,
)
service.wait_for_deployment(show_output=True)

This is what I get from the logs:

service container logs

Checking into the resulting container for the service I can see indeed there is no /runit folder inside /var. There is also no other folders created for the service besides the azureml-app containing my model's files.

I would really appreciate any insights to what's going on here as I have no clue at this point.

user_18428148 · Answer

The error says, it's init() failure and once check whether your model in your workspace is using register() function or not on the model object. If not mentioned, then it will be a model path failure.

def init():
    global model
    model_path = Model.get_model_path('Modelpath')
    model = joblib.load(model_path)

Check the model registry section in this link: https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-deploy-and-where#registermodel

user_11297406 · Answer

With “docker ps -a”, the image created by AML SDK had a command of “runsvdir /var/runit”
a. Article here confirms https://github.com/liupeirong/liupeirong.github.io/tree/master/amlDockerImage 2.Try extracting directory structure from the docker image using undocker project https://github.com/larsks/undocker/
After extracting directory structure of the docker image, find program “runsvdir” and also directory “/var/runit”

Here is sample for custom docker image for “Tensorflow Object Detection” model and deploy same by following this documentation page - https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-deploy-custom-docker-image#use-a-custom-base-image

you can use an existing ACR during workspace creation. Check this doc for details.

AzureML webservice deployment with custom Environment - /var/runit does not exist

2 Answers