'DBT docs generate' does not populate model column-level comments in the catalog

user_4309769 0
Viewed 301

I use dbt-snowflake 1.1.0 with the corresponding dbt-core 1.1.0.

I added documentation for my models in yaml files, i.e.:

> models/stage/docs.yml

version: 2

models:
  - name: raw_weblogs
    description: Web logs of customer interaction, broken down by attribute (bronze). The breakdown is performed using regular expressions.
    columns:
      - name: ip
        description: IP address from which the request reached the server (might be direct customer IP or the address of a VPN/proxy).
...

Although these details show up correctly in the DBT UI when i run dbt docs generate and then dbt docs serve, yet they are not listed in target/catalog.json:

cat target/catalog.json | grep identity

(no results)

According to the DBT documentation, I understand that column comments should be part of catalog.json.

LATER EDIT: I tried running dbt --debug docs generate and it seems that all data is retrieved directly from the target environment (in my case, Snowflake). Looking at the columns of my model in Snowflake, they indeed do NOT have any comments posted on the in Snowflake.

It thus seems to me that the underlying error might be with the fact that dbt run does not correctly persist the column metadata to Snowflake.

dbt
1 Answers

After further investigation, I found out the reason for lacking comments was indeed the fact that the comments are written to catalog.json when running dbt docs generate based on what is received from the database, while dbt docs serve populates the UI by combining information from catalog.json with metadata (in my case, documented column comments) from the local dbt models.

The solution to persist such metadata in the database with dbt run was to add the following DBT configuration:

> dbt_project.yml
models:
  <project>:
    ...
    +persist_docs:
      relation: true
      columns: true
user_43097690
Related