BigQuery : Query error: Column name _PARTITIONDATE is ambiguous when MERGE table

user_6714341 0
Viewed 270

I have a query that will merge the source temp table to target table when there's new row in source that not in target table. the source table used a WITH clause and ROW_NUMBER() to get a non-duplicate row from temp table. I used an ingestion-time partitioned table (day) which mean that during merge I have to specify the column name as from the official doc (Both table have the same table schema)

MERGE mydataset.myfinaltable T 
USING (
 WITH myNonDupSource AS (
        SELECT
        *,
        ROW_NUMBER() OVER (PARTITION BY product, barcode) AS RowNumber
        FROM mySourceTempTable
    )
    SELECT * EXCEPT (RowNumber)
    FROM myNonDupSource
    WHERE RowNumber = 1
) as S
ON
    T.product = S.product
    and  T.barcode= S.barcode
WHEN NOT MATCHED THEN
INSERT (
    col_a,
    col_b,
    col_c,
    product,
    barcode
)
VALUES (
    S.col_a,
    S.col_b,
    S.col_c,
    S.product,
    S.barcode
);

I ran the query and it show query error: Column name _PARTITIONDATE is ambiguous at mydataset.myfinaltable.

At first I try INSERT (col_a,col_b, col_c,product, barcode) ROW; But it produced the same error. Not sure which part that I missed, it shouldn't be ambiguous because I specified the column to be inserted but why the _PARTITIONDATE column still thrown an error

google-bigquery partitioning sql
1 Answers

When you are joining multiple tables in a SQL query and if a column with the same name is present in both of the tables, Bigquery does not know which one to use (unless you explicitly tell so), so it throws the ambiguous column name error.

Bigquery WebUI editor is intelligent enough to highlight the exact row in which the ambiguous column is present (look for the red exclamation on the margin of the sql editor). You can see more documentation.

You need to add the name of the fields instead all the fields (*) like these examples:

MERGE mydataset.myfinaltable T 
USING (
 WITH myNonDupSource AS (
        SELECT
        fieldname1,fieldname2,fieldname3,
        ROW_NUMBER() OVER (PARTITION BY product, barcode) AS RowNumber
        FROM mySourceTempTable
    )
    SELECT fieldname1,fieldname2,fieldname3 EXCEPT (RowNumber)
    FROM myNonDupSource
    WHERE RowNumber = 1
) as S
ON
    T.product = S.product
    and  T.barcode= S.barcode
WHEN NOT MATCHED THEN
INSERT (
    col_a,
    col_b,
    col_c,
    product,
    barcode
)
VALUES (
    S.col_a,
    S.col_b,
    S.col_c,
    S.product,
    S.barcode
);
user_169291450
Related