We define external redshift spectrum table using an yml file. Like the one below.
- name: visitor
description: 'visitor data'
external:
location: "s3://{{ env_var('ENV') }}-temp-raw-s3/{{ s3_prefix({{ env_var('ENV') }}) }}/Visitor"
file_format: parquet
partitions:
- name: year
data_type: VARCHAR(10)
vals: # macro w/ keyword args to generate list of values
macro: dbt.dates_in_range
args:
start_date_str: '{{ var("START_DT") }}'
end_date_str: '{{ var("START_DT") }}'
in_fmt: '%Y-%m-%d'
out_fmt: '%Y'
I would like to change the prefix of the s3 bucket based on the environment variable.
Here is the macro I have tried to change the prefix. I am calling that macro in in the yml file above in line 4 (location).
{% macro s3_prefix(env_var) %}
{% if env_var == "prod" %}
prefix_string = "prefix_prod"
{% else %}
prefix_string = "prefix_non_prod"
{% endif %}
{{ return(prefix_string) }}
{% endmacro %}
I am getting some compilation errors. Is there an easier way to achieve this? Any help would be really appreciated.