Why or how does MLFlow map volumes and ports automatically?

Viewed 11

Yesterday I completed one exercise of the book "Machine Learning Engineering with MLFlow". I was very satisfied, but then I got to think about one thing that was not explained.

When using Docker (unrelated to MLFlow in particular) we build an image and then later when running the image we map ports and volumes in order for the ports to be able to open the ports from inside the container and in order to have the file structure of the host reflected in the container. If we do not do that, that does not happen.

However, now talking about the MLFlow exercise I had the following MLProject file:

name: stockpred

docker_env:
  image:  stockpred

entry_points:
  main:
    command: "python train.py"

I built the image with

docker build -t stockpred .

and then I run

mlflow run .

and after that I had my mlruns folder constructed in my host.

How did MLflow mapp my host volumes to the ones it used inside the container to run train.py??

1 Answers

I investigated further and it seems that when running mlflow run . it automatically calls

docker run --rm -v /home/host/route/mlruns:/mlflow/tmp/mlruns \
-v /home/host/route/mlruns/0/4c277e4e8412a9f708890af939fef/artifacts:/home/host/route/mlruns/0/4c277e4e8412a9f708890af939fef/artifacts \
-e MLFLOW_RUN_ID=4c277e4e8412a9f708890af939fef -e MLFLOW_TRACKING_URI=file:///mlflow/tmp/mlruns \
 -e MLFLOW_EXPERIMENT_ID=0 stockpred:17db901 python train.py' in run with ID '4c277e4e8412a9f708890af939fef' === 

The only strange thing is the second mapping which seems to map a host path in the container path

Related