I tried different solution found on SO. From importing sys to doing different manipulation but I cannot figure it out how to make thing work.
Here my structure folder :
├── __init__.py
├── config
│ └── airflow.cfg
├── dags
│ ├── __init__.py
│ └── inscription.py
├── docker-compose.yml
├── extra
│ ├── __init__.py
│ └── sql_statements.py
├── plugins
│ ├── __init__.py
│ └── operators
│ ├── __init__.py
│ ├── __pycache__
│ ├── has_rows.py
│ └── s3_to_redshift.py
└── requirements.txt
All __ini__.py are empty.
When I import inside dags/inscription.py the code I have in extra/sql_statements.py by doing from extra.sql_statements import *, My IDE (PyCharm) doing scream at me, but when I run the project with docker it fails.
[2019-10-07 07:35:12,286] {{__init__.py:416}} ERROR - Failed to import: /usr/local/airflow/dags/inscription.py
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py", line 413, in process_file
m = imp.load_source(mod_name, filepath)
File "/usr/local/lib/python3.6/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/airflow/dags/inscription.py", line 8, in <module>
from extra.sql_statements import *
ModuleNotFoundError: No module named 'extra'
I don't understand because some other python script I have seems to work that way but I run them without docker. I want to understand what happend here.