Problem

Sagemaker does not load my models from the s3 bucket.

Note: I am passing in sagemaker the S3 URI, where my models should be at. All the models are archived as .tar.gz and they are .onnx models.

Dockerfile

My initial MODEL_BASE_PATH was /opt/ml/model(this is where sagemaker is supposed to store the loaded models from the s3 bucket where you specify the path for).

I modified this path upon recomendation(as /opt/ml/model throws the same exception) to:

    ENV MODEL_BASE_PATH=/opt/ml/models

Problem analysis and code#

I added a few loggers to check if the path in sagemaker of /opt/ml/models actually contains something. Here are the logs, that describe the error (I deleted date and time for them to be easier to read).

/opt/ml: ['models', '.sagemaker_infra']

/opt/ml/models: []

Exception on /ping [GET]

FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/models/<my_model_name>.tar.gz'

"GET /ping HTTP/1.1" 500 265 "-" "AHC/2.0"

"GET /models HTTP/1.1" 404 2 "-" "AHC/2.0"

As you can see from the above logs, /opt/ml/models is empty, so the models from the s3 bucket are not loaded there. So I get an exception from the code below, that points out the model could not be loaded as there is no such file or directory(Because nothing exists at that location).

    @app.route("/ping", methods=["GET"])
    def ping():
    """Determine if the container is working and healthy. In this sample container, we declare
    it healthy if we can load the model successfully."""
        health = ScoringService.get_model() is not None  # You can insert a health check here

        status = 200 if health else 404
        return flask.Response(response="\n", status=status, mimetype="application/json")

"ScoringService.get_model()" Is supposed to return a string with the path of the model I am using. This works on a simultaion of the sagemaker structure on my local computer with a request from Postman.

More on the project structure: https://sagemaker-workshop.com/custom/containers.html

My project structure:

    project_name/
        Dockerfile
        build_and_push.sh
        inference/
            nginx.conf
            predictor.py
            serve
            wsgi.py

Note: I have left out the file 'train', as I will not be adding further models to my application.

Endnote: Any insight on what I am doing wrong is really helpful.