Recently I'm receiving these errors, trying to modify query_job.result parameters page_size, timeout
and also setting Retry handler with deadline=240.
however somewhere at the background of bigquery's library there is request with timeout=60 I can't change.
I've been using this code for 6 month and this week is the first time I got this error.
I tried to add the bqstorageclient option, pandas gbq downloader, and to_dataframe with bqstorageclient
How should I configure the query to get all the data?
import google.api_core.retry.Retry
bqstorageclient = bigquery_storage_v1beta1.BigQueryStorageClient(
credentials=credentials
)
r = Retry(deadline=240)
query_job = client.query("select * from `table_name`") # client(bigquery client)
# tried without any params, with page_size=1,100,500,1000 and timeout=60,120,240
rows = query_job.result(timeout=240, page_size=500, retry=r)
data = [r for r in rows]
# df = pd.read_gbq("select * from `table_name`", use_bqstorage_api=True)
# df = rows.to_dataframe(bqstorage_client=bqstorageclient)
thanks in advance