Clarification about streaming buffer in big query

user_9216423 0
Viewed 681

I just started to discover Big Query in GCP for my learning purpose. so I created two tables and tried to insert, delete and update queries by using python API.

I'm able to update a table called table_1 any time using the below query

UPDATE *****.*****.table_1 SET col_1 = 'value_1', col_2 = 'value_2' WHERE col3 = 'value_3'

and it returns This statement modified 2 rows in ****:****.Projects.

But when I try to update the table called table_2 using a query in the same way it returns

UPDATE or DELETE statement over table ***.***.table_2 would affect rows in the streaming buffer, which is not supported

so I created tables and perform operations in the same way, my problem is why I'm getting this error only for the table_2

Thank you

google-bigquery
1 Answers

Streamed data is available for real-time analysis within a few seconds of the first streaming insertion into a table but it can take up to 90 minutes to become available for copy/export and other operations like UPDATE or DELETE. You probably have to wait up to 90 minutes so all buffers are persisted on the cluster. You can check the ‘tables.get’ response for a section named ‘streamingBuffer’ to check whether the table has a streaming buffer or not. If you have used a load job to create the table, you won't have a streaming buffer, but probably you streamed some values to it.

You can also refer to this documentation [1] for more information [1] https://cloud.google.com/bigquery/streaming-data-into-bigquery

user_157471080
Related