Python dependencies and Docker and Crontab

user_6692555 0
Viewed 2208

I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.

I have a Dockerfile like this:

FROM python:3-onbuild

# Install cron and dependencies
RUN apt-get update && apt-get -y install cron

# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src

# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages

# Create the log file to be able to run tail
RUN touch /var/log/cron.log

# Run the command on container startup
CMD ["cron", "-f"]

And a crontab file like this:

* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2

This executes correctly with no problems when I start the container. The first line of my Python file is import redis, which causes the script to fail, saying that the module cannot be found.

The interesting thing is that when I manually execute docker exec -it [container_name] python test.py, everything works exactly as expected.

What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?

docker redis cron python
2 Answers

With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN commands which:

  1. Establish the virtualenv
  2. Activate it
  3. call pip install -r requirements.txt

Here's an example from a script I use in dev:

pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;
user_105539760

cron runs commands in your home directory, not in the WORKDIR from your Dockerfile. docker exec -it... will run the specified command from the WORKDIR, which is /src in your case. My guess is you install your requirements to a location that isn't included in your PYTHONPATH, but is visible from your /src directory? As @Klaus D. pointed out, it would be helpful to know where your requirements are installed... but if my suspicion is correct, there are two easy solutions:

  1. change your cron job command to cd /src && python ...
  2. Explicitly set your PYTHONPATH to include wherever your requirements are being installed

To further debug this, do docker exec -it your-container-name bash, and then do cd ~ to change to your home directory because that is where cron jobs run from. Once there, try running your script, and if that doesn't work, try echo $PYTHONPATH to start debugging your import issue.

user_40886420
Related