In dbt source configuration, can I map one Source name to multiple schema names?

user_18147264 0
Viewed 1094

I am using dbt sources to describe ingested data from frontline production databases.

What makes sense in our setting is to have one source block to contain more than one physical schemas (and tables in them). It doesn't look like this is possible in current dbt version. Am I missing something? And how do others solve this problem?

In our case, we use Fivetran to import tables from different production sources. For legacy reasons, we now have multiple schemas in our BigQuery database, even though logically they are the same "source".

Again, what's a better solution to this problem?

dbt
1 Answers

I think since the raw tables are in different schemas, you will have to define them under separate sources.

However, you might take advantage of using tags in order to reap similar benefits to having them in the same source.

Ex.

sources:
  - salesforce
    database: fivetran
    schema: prod
    tags:
      - sfdc

  - salesforce_legacy
    database: fivetran
    schema: legacy
    tags:
      - sfdc

I assume you want to map these schemas to a single source, so you can use model selection logic like dbt run --select source:salesforce+.

Tags are one option to help accomplish the same, as in: dbt run --select tag:sfdc+

user_82861690
Related