I have a table in Google BigQuery that I access and modify in Python using the pandas functions read_gbq and to_gbq. The problem is that appending 100,000 lines takes about 150 seconds while appending 1 line takes about 40 seconds. I would like to update a value in the table rather than append a line, is there a way to update a value in the table using python that is very fast, or faster than 40 seconds?
Python: How to update a value in Google BigQuery in less than 40 seconds?
Viewed 6550
2 Answers
With the google-cloud library update, you can now do as follows .
from google.cloud import bigquery
client = bigquery.Client(credentials=credentials, project=project)
query =
"""
(
UPDATE `dataset.table` SET field_1 = '3' WHERE field_2 = '1
)
"""
job_config = bigquery.QueryJobConfig(use_legacy_sql=False)
query_job = client.query(query, job_config=job_config)
print("Updated query")
for row in query_job:
print(row)
Related