Apache Airflow: Pass variable in jinja include

user_12742892 0
Viewed 941

I am currently using following statement in Apache airflow

s10_test_load = bigquery.BigQueryInsertJobOperator(
    task_id="10_test_load",
    configuration={
            "query": {
                        "query": "{% include './scripts/10_test_load.sql' %}",                           
                        "useLegacySql": False,
            },                  
    },
)

This statement is working fine and is executing sql from the script file.

I would like dynamically use the file name from a variable like following

file_name='./scripts/10_test_load.sql'

s10_test_load = bigquery.BigQueryInsertJobOperator(
    task_id="10_test_load",
    configuration={
            "query": {
                        "query": "{% include file_name %}",                          
                        "useLegacySql": False,
            },                  
    },
)

How I can pass a python variable inside jinja include?

google-cloud-composer airflow google-bigquery jinja2 python
1 Answers

You could use f-string formating and add an extra pair of curly braces like this:

file_name='bq/queries/10_test_load.sql'

t2 = BigQueryInsertJobOperator(
   task_id="10_test_load",
    configuration={
        "query": {
            "query": f"{{% include '{file_name}' %}}",                          
            "useLegacySql": False,
        },                  
    },             
)

This gets rendered the desired way: enter image description here

user_79031590
Related