airflow not recognize local directory ModuleNotFoundError: No module named

user_1928414 0
Viewed 2738

my project structure looks like :

my_project
 -dags
 -config

however on airflow dashboard I see an error Broken DAG pointing to this line : from config.setup_configs import somemethod

and yields this err:

Broken DAG: [/usr/local/airflow/dags/airflow_foo.py] No module named 'config'

although the directory exists

airflow
2 Answers

According to documentation Airflow has, by default, three directories to path

  • AIRFLOW_HOME/dags
  • AIRFLOW_HOME/config
  • AIRFLOW_HOME/plugins

Any other path has to be added to system path, as described in airflow module management

For sake of simplicity, I added my module mymodule.py to AIRFLOW_HOME/plugins and I can import them successfully.

from mymodule import my_method

So, in your case, if you rename configurations to plugins and update import statement into DAG,

from setup_configs import somemethod

it should work.

user_111333890

You need to move the config directory into the dags folder and create an empty __init__.py file within the config folder. Then, it should work.

user_109055350
Related