I use subprocess.popen in python3 to put files or make directories in hdfs. it runs accurately using python3 on the Linux shell. but as I use crontab to run the code, I get "FileNotFoundError: [Errno 2] No such file or directory: 'hdfs': 'hdfs'" Error on my log file.
make_dir = subprocess.Popen(['hdfs','dfs','-mkdir' , '-p' , hdfs_path])
I had similar problem when my script executed by Airflow in the same machine, Airflow did not load my environment variables. I had to manually set the environment variables in code. you can find how to do that in code here. just update PATH to include Hadoop binaries path. also it is suggested that you can load environment variables in cron job script. like:
#!/bin/bash
source $HOME/.bash_profile
some_other_cmd