apache-airflow-providers-google in MWAA

user_5759056 0
Viewed 572

I tried to install apache-airflow-providers-google in MWAA but it seems that Airflow does not pick the library:

    Broken DAG: [/usr/local/airflow/dags/mi-framework/jobs/google_analytics/main.py] Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/airflow/dags/mi-framework/jobs/google_analytics/main.py", line 8, in <module>
    from airflow.providers.google import *
ModuleNotFoundError: No module named 'airflow.providers.google'

I have the following requirements.txt file:

apache-airflow[postgres]==2.0.2
apache-airflow[mysql]==2.0.2
apache-airflow-providers-google==6.0.0
sqlalchemy==1.3.24
google-api-python-client==2.27.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.6
sentry-sdk

What do I do wrong?

mwaa airflow google-cloud-platform
3 Answers
user_91827320

The resolution was, upgrading MWAA to 2.2.2

user_57590560

In my case, using below requirements made:

  • MWAA work with Bigquery operators
  • Allowed me to add a google cloud connection from the Airflow UI

Make sure to have the same versions! (I am using MWAA 2.2.2)

apache-airflow[postgres]==2.2.2
apache-airflow[mysql]==2.2.2
apache-airflow-providers-google==6.0.0
sqlalchemy==1.3.24
google-api-python-client==1.12.8
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.6

Hope it can help!

user_112262140
Related