Python script in docker can't find module which can be found correctly without docker

user_10025767 0
Viewed 677

I have the following project tree:

    ├── Dockerfile
    ├── modules
│   ├── __init__.py
│   ├── scripts_A
│   │   ├── file_1.py
│   │   ├── __init__.py
│   │   └── __pycache__
│   └── scripts_B
│       ├── file_2.py
│       ├── __init__.py
│       └── __pycache__

The Dockerfile:

FROM python:3.7-slim-buster

WORKDIR /app

RUN apt-get update

RUN mkdir /modules

COPY script.py  /app/
COPY ./modules/* /app/modules/ 
RUN export PYTHONPATH=/app/

RUN dir

CMD [ "python", "script.py" ]

and the files content is as follows:

#script.py
from modules.scripts_B.file_2 import *

def hello_world():
    external_print()

if __name__ == '__main__':
    hello_world()


#file_1.py
def my_print():
    print("Hi")


#file_2.py
from modules.scripts_A.file_1 import *

def external_print():
    my_print()

If I launch the python script.py on my pc everything is correct, but when I run the same script via docker it raises an error:

Traceback (most recent call last):
  File "script.py", line 2, in <module>
    from modules.scripts_B.file_2 import *
ModuleNotFoundError: No module named 'modules.scripts_B'

Does anybody see where the bug origins from?

UPD I added __init__, nothing has changed. I added inits files everywhere problem is still present

docker python
2 Answers

You have two issues:

  1. You are missing the __init__.py for each package in your python code structure. You should have something similar to this:
├── Dockerfile
├── modules
│   ├── __init__.py
│   ├── scripts_A
│   │   ├── file_1.py
│   │   ├── __init__.py
│   └── scripts_B
│       ├── file_2.py
│       ├── __init__.py
└── script.py
  1. In your Dockerfile, you are creating the directory "/modules" and copying the files in the wrong way. You should change your Dockerfile for this and it should work fine:
FROM python:3.7-slim-buster

WORKDIR /app

RUN apt-get update

RUN mkdir /app/modules

COPY script.py  /app/
COPY ./modules /app/modules/ 
RUN export PYTHONPATH=/app/

RUN dir

CMD [ "python", "script.py" ]

Here is the problem:

COPY ./modules/* /app/modules/:

root@1be2aa295dbf:/app# ls -lah /app/modules/
total 28K
drwxr-xr-x 2 root root 4.0K Sep 29 20:46 .
drwxr-xr-x 1 root root 4.0K Sep 29 20:46 ..
-rw-rw-r-- 1 root root    0 Sep 29 20:44 __init__.py
-rw-rw-r-- 1 root root   32 Sep 29 20:42 file_1.py
-rw-rw-r-- 1 root root   76 Sep 29 20:44 file_2.py

As you can note, when copied the files aren't structured as modules.

With COPY ./modules /app/modules/ the files are copied into the correct sub-directory structure.

root@49d94b8f0941:/app# ls -lah /app/modules/
total 20K
drwxr-xr-x 1 root root 4.0K Sep 29 20:56 .
drwxr-xr-x 1 root root 4.0K Sep 29 20:49 ..
-rw-rw-r-- 1 root root    0 Sep 29 20:44 __init__.py
-rw-rw-r-- 1 root root  146 Sep 29 20:44 __init__.pyc
drwxrwxr-x 2 root root 4.0K Sep 29 20:45 scripts_A
drwxrwxr-x 2 root root 4.0K Sep 29 20:45 scripts_B
user_169937280

Module is any Python file and a package is a folder with (at least) __init__.py file. You are missing those in the folder, thus no module for you when importing. Add them to the image.

The __init__.py files are required to make Python treat directories containing the file as packages. This prevents directories with a common name, such as string, unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later. (source)

...
├── modules
│   ├── __init__.py
│   ├── scripts_A
│   │   ├── __init__.py
...
│   └── scripts_B
│   │   ├── __init__.py
...

Dockerfile:

FROM python:alpine
RUN mkdir modules && mkdir modules/package_1 && \
    echo 'print("Hello")' > modules/package_1/__init__.py && \
    echo 'from modules.package_1 import *' > script.py
RUN python script.py

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM python:alpine
 ---> 03c59395ddea
Step 2/3 : RUN mkdir modules && mkdir modules/package_1 &&     echo 'print("Hello")' > modules/package_1/__init__.py &&     echo 'from modules.package_1 import *' > script.py
 ---> Running in 99e2f69bbe0c
Removing intermediate container 99e2f69bbe0c
 ---> 82ae26cd9103
Step 3/3 : RUN python script.py
 ---> Running in c8b9bc76403b
Hello
Removing intermediate container c8b9bc76403b
 ---> 393bafc59987
Successfully built 393bafc59987
user_59940410
Related