Permission denied when using BigQuery public data set as a source

user_8677355 0
Viewed 1999

When setting up a source that points to a Big Query public data set and using the source() function to reference it in my dbt model, I get this error on dbt run:

Database Error
  Access Denied: Table bigquery-public-data:INFORMATION_SCHEMA.SCHEMATA: User does not have permission to query table bigquery-public-data:INFORMATION_SCHEMA.SCHEMATA.

It appears to be trying to query the information schema from the public data set. Is it possible to use a public data set as a source in my dbt project?

dbt google-bigquery
2 Answers

This is expected on earlier versions of dbt — it has to do with the fact that you can't grant permissions on bigquery-public-data.INFORMATION_SCHEMA.SCHEMATA to the public (related SO question).

Fortunately, this issue was resolved in dbt v0.17.0 (relevant PR)— try upgrading to that version and the error should disappear!

user_29308730

I had this issue when my data processing location was in the EU / in the profiles.yml file changed it to US (also of course will need a dataset location in US) and bingo it ran as expected in the tutorial. I had it in Version DBT 1.x

user_178348590
Related