Creating or updating a table from a select statement in Google BigQuery

Viewed 1030

By using scripting with a variable to optimize data processing for a query Google BigQuery optimization with subquery in WHERE clause, the ability to specify a destination was lost.

Modeling a solution described in an answer to the question Get a Big Query script to output a table, generates a 'configuration.query.writeDisposition cannot be set for scripts' error.

What are options to address the error, where the data are needed to be written (overwritten) to a table? This is necessary to allow for post-processing and writing (appending) to another table for reporting.

The script is given below.

DECLARE start_date STRING;
SET start_date = (SELECT MAX(date) FROM `my-project-12345.dataset.my-data-table`);
CREATE OR REPLACE TABLE `my-project-12345.dataset.my-staging-table`
AS SELECT
  GA.date, 
  GA.field1, 
  hits.field2, 
  hits.field3
FROM 
  `my-project-12345.dataset.ga_sessions_*` AS GA, UNNEST(GA.hits) AS hits
WHERE 
  hits.type IN ('PAGE', 'EVENT')
  AND hits.field0 = 'some value'
  AND _TABLE_SUFFIX > start_date
  AND _TABLE_SUFFIX < '20201025' 

Query settings

1 Answers

Looks like that Console UI is reusing whatever settings for Destination was previously used in Query settings panel

enter image description here

So, looks like you need to make sure you disable Destination

enter image description here

For this you need to make sure you have any simple query (not a script) so above settings are visible - do needed changes and don't forget to save 'em!

After this done - your script should go!

Related