Cron and virtualenv

Don't look any further:

0 3 * * * /usr/bin/env bash -c 'cd /home/user/project && source /home/user/project/env/bin/activate && ./manage.py command arg' > /dev/null 2>&1

Generic approach:

* * * * * /usr/bin/env bash -c 'YOUR_COMMAND_HERE' > /dev/null 2>&1

The beauty about this is you DO NOT need to change the SHELL variable for crontab from sh to bash

I am sorry for that nth answer but I checked the answers and there is really simpler and neater.

Long story short

Use the python binary of your venv in your cron :

0 3 * * * /home/user/project/env/bin/python /home/user/project/manage.py 

Long story

We activate the virtual environment when we want to set the current shell with the python config of that specific virtual environment(that is binaries and modules of that).
It is relevant to work with the current shell : execute multiple python commands on the current shell without the need to reference the full python path of the venv.
In the frame of a cron or even a bash, which value to activate the environment ? Besides I read in some answers some references to bash rather than sh or still to define a wrapper to call the Python code. But why the hell should we bother with these ?

I repeat, just do it :

0 3 * * * /home/user/project/env/bin/python /home/user/project/manage.py 

The documentation confirms that :

You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.

This is a simple way that keeps the crontab command very similar to regular command (tested in Ubuntu 18.04). Some key notes to keep in mind:

• You can use the . command instead of source. (crontab uses sh by default, not bash, so it doesn't have source.)
• ~ and $variables are expanded in crontab commands. (It's only crontab environment statements that don't do variable expansion.)

Here are examples if you have a file ~/myproject/main.py:

* * * * * cd ~/myproject && . .venv/bin/activate && python main.py > /tmp/out1 2>&1

You could also directly call the specific path of the python in the venv directory, then you don't need to call activate.

* * * * * ~/myproject/.venv/bin/python ~/myproject/main.py > /tmp/out2 2>&1

The downside of that is you would need to specify the project path twice, which makes maintenance trickier. To avoid that, you could use a shell variable so you only specify the project path once:

* * * * * project_dir=~/myproject ; $project_dir/.venv/bin/python $project_dir/main.py > /tmp/out3 2>&1

I'd like to add this because I spent some time solving the issue and did not find an answer here for combination of variables usage in cron and virtualenv. So maybe it'll help someone.

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DIR_SMTH="cd /smth"
VENV=". venv/bin/activate"
CMD="some_python_bin do_something"
# m h  dom mon dow   command
0 * * * * $DIR_SMTH && $VENV && $CMD -k2 some_target >> /tmp/crontest.log 2>&1

It did not work well when it was configured like

DIR_SMTH="cd /smth && . venv/bin/activate"

Thanks @davidwinterbottom, @reed-sandberg and @mkb for giving the right direction. The accepted answer actually works fine until your python need to run a script which have to run another python binary from venv/bin directory.

If you're on python and using a Conda Virtual Environment where your python script contains the shebang #!/usr/bin/env python the following works:

* * * * * cd /home/user/project && /home/user/anaconda3/envs/envname/bin/python script.py 2>&1

Additionally, if you want to capture any outputs in your script (e.g. print, errors, etc) you can use the following:

* * * * * cd /home/user/project && /home/user/anaconda3/envs/envname/bin/python script.py >> /home/user/folder/script_name.log 2>&1

python script

from datetime import datetime                                                                                                                                                                
import boto   # check wheather its taking the virtualenv or not                                                                                                                                                                        
import sys                                                                                                                                                                                   
param1=sys.argv[1]     #Param                                                                                                                                                                                                                                                                                                                                                                    
myFile = open('appendtxt.txt', 'a')                                                                                                                                                      
myFile.write('\nAccessed on ' + param1+str(datetime.now())) 

Cron command

 */1 * * * *  cd /Workspace/testcron/ && /Workspace/testcron/venvcron/bin/python3  /Workspace/testcron/testcronwithparam.py param  

In above command

• */1 * * * * - Execute every one minte
• cd /Workspace/testcron/ - Path of the python script
• /Workspace/testcron/venvcron/bin/python3 - Virtualenv path
• Workspace/testcron/testcronwithparam.py - File path
• param - parameter

I've added the following script as manage.sh inside my Django project, it sources the virtualenv and then runs the manage.py script with whatever arguments you pass to it. It makes it very easy in general to run commands inside the virtualenv (cron, systemd units, basically anywhere):

#! /bin/bash

# this is a convenience script that first sources the venv (assumed to be in
# ../venv) and then executes manage.py with whatever arguments you supply the
# script with. this is useful if you need to execute the manage.py from
# somewhere where the venv isn't sourced (e.g. system scripts)

# get the script's location
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

# source venv <- UPDATE THE PATH HERE WITH YOUR VENV's PATH
source $DIR/../venv/bin/activate

# run manage.py script
$DIR/manage.py "$@"

Then in your cron entry you can just run:

0 3 * * * /home/user/project/manage.sh command arg

Just remember that you need to make the manage.sh script executable

I had the same issue and spent a lot of time solving that. None of the solutions here helped me, so I'm sharing what worked for me:

1. Open a new file "pick_name.sh" open it inside of your project directory.
2. Inside the "pick_name.sh" file, write and save the following lines:

#!/bin/bash
source /YOUR_VIRTUAL_ENV_PATH/bin/activate
export PYTHONPATH="${PYTHONPATH}:/PATH_TO_CUSTOM_MODULE_YOU_CREATED**OPTIONAL**"
export PYTHONPATH="${PYTHONPATH}:/PATH_TO_ANOTHER_CUSTOM_MODULE_YOU_CREATED**OPTIONAL**"
cd /PATH_TO_DIR_STORING_FILE_NAME.PY python file_name.py

3. Go to /var/spool/cron/crontabs (or to where your cron management file sits) and open the 'root' file.
4. Add these lines to the root file which's inside the crontab folder:

# m h  dom mon dow   command
* * * * * /PATH_TO_DIR_WHERE_PICK_NAME.SH_SITS/pick_name.sh >> /YOUR_PROJECT_DIR/cron_output.txt 2>&1

Notes:

• This command (section 4.) will run the "pick_name.sh" file. In this example it runs every minute, so make sure you change it according to your needs. It writes all logs to a log file called "cron_ouput". No need to create the file before, it will be created automatically.
• Make sure to replace all paths (I wrote them in capital letters) to your paths.
• You can change file names, if so, make sure to change it in all appearances in my instructions to avoid errors.
• If you want to add another py file to run by cron, you need to add it to the "pick_nam.sh" file* not to the cron. Simply duplicate section 2. lines in the "pick_nam.sh" but without the "#!/bin/bash" part. Then, every time the cron will run "pick_name.sh" it will run all the files you specified inside of it.
• Make sure to restart cron after changes, it could have saved me a lot of debugging time, use this command:

systemctl restart cron

Since a cron executes in its own minimal sh environment, here's what I do to run Python scripts in a virtual environment:

* * * * * . ~/.bash_profile; . ~/path/to/venv/bin/activate; python ~/path/to/script.py

(Note: if . ~/.bash_profile doesn't work for you, then try . ~/.bashrc or . ~/.profile depending on how your server is set up.)

This loads your bash shell environment, then activates your Python virtual environment, essentially leaving you with the same setup you tested your scripts in.

No need to define environment variables in crontab and no need to modify your existing scripts.

Add the path of the Python installation in the venv but don't activate the environment.

* * * * * /HDD1/shritam_kumar/VENOM/venv/bin/python /HDD1/shritam_kumar/Projects/Voelkner-DE/schedule_product_BA.py

It's that simple.

If you are a MacOS user like me, you can check the crontab error message at /var/mail/{username} file. like this

tail /var/mail/{username}

If there is an "operation not permitted" error, maybe you have to add cron to the Full Disk Access apps (Security & Privacy > Privacy > Full Disk Access apps/execs).

And click + button, go to /usr/sbin, double click the cron file. then it will fix the "not permitted" error. detailed steps

And this is my code：

0 19 * * * cd /Users/user/Desktop/Project && source /Users/user/Desktop/Project/venv/bin/activate && python command arg

This is a solution that has worked well for me.

source /root/miniconda3/etc/profile.d/conda.sh && \
conda activate <your_env> && \
python <your_application> &

I am using miniconda with Conda version 4.7.12 on a Ubuntu 18.04.3 LTS.

I am able to place the above inside a script and run it via crontab as well without any trouble.

This will also work on crontab -e

* */5 * * * cd /home/project && sudo /home/project/venv/bin/python scripte.py

I had this same issue:

I had written a custom django command to check for geodjango position coordinates inside of geodjango polygons and had trouble automating the task to run, however using this command with crontab worked for me:

* * * * * ./home/project/locations/locations.sh >> /var/log/locations.log 2>&1