Making life easier,
this project is great for running jobs in pySpark by Airflow or Jupyter notebooks using Docker. Unfortunately, the Airflow part fails.
I guess it is because the SparkSubmitOperator can´t find the spark binaries but in the settings, everthing looks fine.
There is an open issue with details for that. I hope we can solve it together since I´m new to Docker-Compose and probably miss some relevant points.
Thanks for your support.