DBT set variable using macros

WinstonKyu 0
Viewed 39

my goal is to get the last 2 dates from the tables and run insert_overwrite to load incremental on a large table. I am trying to set a variable inside the model by calling on the macros I wrote. The SQL query is in BigQuery.

I get an error message.

'None' has no attribute 'table'

inside model

{% set dates = get_last_two_dates('window_start',source('raw.event','tmp')) %}

macros

{% macro get_last_two_dates(target_column_name, target_table = this) %}

{% set query %}
select string_agg(format('%T',target_date),',') target_date_string
from (
SELECT distinct date({{ target_column_name }}) target_date
FROM {{ target_table }}
order by 1 desc
LIMIT 2
) a
{% endset %}

{% set max_value = run_query(query).columns[0][0] %}
{% do return(max_value) %}

{% endmacro %}

Thanks in advance. let me know if you have any other questions.

dbt jinja2 google-bigquery
1 Answers

You probably need to wrap {% set max_value ... %} with an {% if execute %} block:

{% macro get_last_two_dates(target_column_name, target_table = this) %}

{% set query %}
select string_agg(format('%T',target_date),',') target_date_string
from (
SELECT distinct date({{ target_column_name }}) target_date
FROM {{ target_table }}
order by 1 desc
LIMIT 2
) a
{% endset %}

{% if execute %}
{% set max_value = run_query(query).columns[0][0] %}
{% else %}
{% set max_value = "" %}
{% endif %}
{% do return(max_value) %}

{% endmacro %}

The reason for this is that your macro actually gets run twice -- once when dbt is scanning all of the models to build the DAG, and a second time when the model is actually run. execute is only true for this second pass.

tconbeer0
Related