Big Query Table not being partitioned with firestore Export Collections to BigQuery extension

Viewed 96

enter image description here

I am using Export Collections to BigQuery extension with firestore to export data to the bigquery.I can export data to the big query but the table is not getting partitioned even I have enabled the table partitioned with extension while configuring extension at the creation time.

1 Answers

I had the same issue today when exporting data from Firestore to BigQuery and doing a historical load with npx firebaseextensions/fs-bq-import-collection command.

These are my findings. If you configure the extension to have partitioning based on the ingestion time, you have to wait for a first new record that will trigger that streaming extension and create a partitioned table.

If you just create an extension and immediately execute a historical load, the table created won't be partitioned. I tested this 3 times today.

A question for you -- did you execute a historical load or imported only new records?

Related