I have seen people creating JOB_flow_overrides in creating EMR. I have set up all those master and slave nodes required in emr_default (extra), DO I still need it? In what cases will use this parameter?
1 Answers
job_flow_overrides is used to create a new emr cluster every time the dag is ran.
https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/operators/emr.html
Related