We plan to have one common AWS MWAA cluster in us west region which triggers emr jobs in different regions.
Is there a way to trigger a emr job in one region by an airflow job running in another region? Is it ever possible?
1 Answers
Airflow doesn't care where it's being deployed. Think if you would have deployed Airflow in Google, Azure or any other service - that wouldn't have block you from triggering EMR jobs in any AWS region you wish.
Triggering EMR jobs can be done in any region you wish. It has nothing to do with the region MWAA is set on.
You'll need to define Aws connection for each relevant region and utilize the relevant connection with the operator.
Related