airflow BigqueryOperator is not working xcom_pull

user_17462031 0
Viewed 150
Bq = bigquery_operator.BigQueryOperator(
    task_id="Bq",
    bigquery_conn_id='xxxxxxxxxxx',
    use_legacy_sql=False,
    sql='aaaaaaa.sql',
    params={"project_id": project_id,
            "dataset_table_name": bq_dataset_table_name,
            "target_date": '{{ti.xcom_pull(task_ids="date")["date"]}}'},
)

　If you do this, the xcom_pull will not be expanded and will be passed to SQL. Is there a solution?

airflow
1 Answers

params is not templated field. There is no need to pass xcoms with params. You can use it directly in aaaaaaa.sql.

I assume that in aaaaaaa.sql you are referencing with {{ params.target_date }}. To solve your issue replace just replace it with {{ ti.xcom_pull(task_ids="date")["date"] }}

Example:

Bq = bigquery_operator.BigQueryOperator(
    task_id="Bq",
    bigquery_conn_id='xxxxxxxxxxx',
    use_legacy_sql=False,
    sql='aaaaaaa.sql',
    params={"project_id": project_id,
            "dataset_table_name": bq_dataset_table_name,
)

and in aaaaaaa.sql :

SELECT *
FROM {{ti.xcom_pull(task_ids="date")["date"]}}
user_146244090
Related