Can you get data from a Big Query table to an outside stream

Viewed 37

Because of companies policies, we have a lot of information that we need as input inserted into a BigQuery table that we need to SELECT from.

My problem is that doing a select directly into this table and trying to run a process (a virtual machine, etc) is prone to errors and reworking. If my process stops, I need to run the query again and reprocess everything.

Is there a way to export data from Big Query to a Kinesis-like stream (I'm more familiar with AWS)?

1 Answers

DataFlow + PubSub seems to be the way to go for this kind of issue.

Thank you jamiet!

Related