Python Script not working as expected when run by Crontab

user_12224888 0
Viewed 149

I have to run a specific python script which is requesting some data over the REST API every night at 3. For that I have enabled a crontab:

0 3 * * * python3 /home/pi/path/to/script.py >> /home/pi/logfile.txt 2>&1

When I execute the script on its own everything is working as expected, but when the cron starts the script I get errors that the nameserver can not resolve my request:

 File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 57, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

I verified that the script is executed by the same user (pi).

Im behind a company proxy but so far I had no problems with that when launching it manually. Can someone point me to the difference in the execution. I thougt it has to be the same.

  • Hardware: Pi4 4GB
  • OS: Linux raspi 5.10.11-v7l+ #1399 SMP Thu Jan 28 12:09:48 GMT 2021 armv7l GNU/Linux
  • Software:
    • Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux
    • requests==2.21.0

UPDATE:

I found out that the problem is that the global proxy settings are not active when launching my script with the crontab.

I could make it work when i call a bash script from cron that is setting the proxy and than calling the script:

#!/bin/bash
export http_proxy="http://proxyip:proxyport"
export https_proxy="http://proxyip:proxyport"
python3 /home/pi/path/to/script.py >> /home/pi/logfile.txt 2>&1
python-3.x rest cron python
1 Answers

As stated in the update above the problem was the proxy. The global settings that are known by pyton when manually executed are not known when launching it from crontab. The solution for this problem is to add the proxy config to the crontab file:

HTTP_PROXY=http://proxyip:proxyport
HTTPS_PROXY=http://proxyip:proxyport
0 3 * * * python3 /home/pi/path/to/script.py >> /home/pi/logfile.txt 2>&1

BUT: This only seems to work within the file directly called form crontab. In my case I call another bash script from the python file that is also not able to resolve the address. For that another call of:

export http_proxy="http://proxyip:proxyport"
export https_proxy="http://proxyip:proxyport"

is needed before execution!

Thanks to farbiondrive for giving the hint for the solution!

user_122248880
Related