BigQuery Clustered Tables: How to Create Multiple Clusters

Viewed 1849

My BigQuery table is commonly queried with different combinations of "where" conditions across 1 or more common columns, say across columns A, B, C (not in order). Hence, I would like to add individual clusters for columns A, B, and C respectively.

How can I create multiple clusters for BigQuery tables? (Similar to how multiple indexes can be created on a traditional rdbms table)

2 Answers

Multiple clustering is allowed (but it is hierarchical,you cluster by specific field and then it is subclustered on the following, etc).

At the same time, clustering is only allowed for partitioned tables.

You can find the corresponding documentation here

Upon viewing some comments and pages it appears that there are no ways to have multiple independent clusters (vs how multiple indexes can be created on a traditional rdbms) on a single bigquery table.

This is because clusters pretty much just sort the data blocks of that table as per docs:

When data is written to a clustered table by a query job or a load job, BigQuery sorts the data using the values in the clustering columns. These values are used to organize the data into multiple blocks in BigQuery storage. When you submit a query that contains a clause that filters data based on the clustering columns, BigQuery uses the sorted blocks to eliminate scans of unnecessary data.

Hence, it appears that there is no way of applying multiple sorting logic for each independent cluster on the same set of data, and so what I require appears to be impossible as of now.

Related