Google BigQuery Python Client using the wrong credentials

Viewed 616

I am getting a "Not Authorized" error when I try to query a table in BigQuery. I've set:

os.environ["GOOGLE_APPLICATION_CREDENTIALS"]=<path>

Then used

client = bigquery.Client()

When I issue the query using

client.query("SELECT * from `don.book3`;")

I get the response:

Forbidden: 403 Access Denied: Table <project-name>:don.book3: User does not have permission to query table <project>:don.book3

OK, so I checked the service email address with

client.get_service_account_email()

and found out it was transmitting the wrong address -- the one for the default Compute Engine service account, not the one for the query service account I'd set up. OK, so it's ignoring the environment variable that I've set. Let's try again with:

from google.oauth2 import service_account

# TODO(developer): Set key_path to the path to the service account key
#                  file.
# key_path = "path/to/service_account.json"
key_path = <path>

credentials = service_account.Credentials.from_service_account_file(
    key_path
)

client = bigquery.Client(credentials=credentials, project=credentials.project_id)

Same error. So again I try:

client.get_service_account_email()

And got back the wrong service account address. So I thought the credentials must be wrong, and tried:

credentials.service_account_email And got back the right address! So bigquery Client is shrugging off all attempts to set the credentials. The last thing I tried was to make sure that the credentials actually had the authorization to do what I want, so I set GOOGLE_APPLICATION_CREDENTIALS in a shell, checked to make sure it was properly set in the environment, and then from the command line ran:

bq query "select * from don.book3;"

And it worked fine. So all of this experimentation established:

  1. The credentials were configured correctly
  2. The service account did have permission to run the query
  3. The bigquery Python library was using the wrong credentials.

I'm using Python 3.8 and the following libraries:

  • google-auth 1.32.0
  • google-auth-oauthlib 0.4.4
  • google-cloud-bigquery 2.20.0

Has anyone run into this/knows a fix?

0 Answers
Related