Set Google Cloud connection in Airflow using env vars

user_3064736 0
Viewed 704

How can you set Google Cloud / BigQuery connection parameters in Airflow by using Environment Variables; and so avoiding requiring logging into the UI or running a script?

airflow
1 Answers

Assuming the project is in the env var GOOGLE_CLOUD_PROJECT, you can set these

AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT=;?extra__google_cloud_platform__project=$GOOGLE_CLOUD_PROJECT
AIRFLOW_CONN_BIGQUERY_DEFAULT=;?extra__google_cloud_platform__project=$GOOGLE_CLOUD_PROJECT

Ideally Airflow would allow a default project of None, and the underlying libraries would handle this

user_30647360
Related