I have the following query using by DBT to transform one of my datasets.
with source as (
select * from {{source("datalake", "customer_address")}}
),
step_0 as (
select
cast("id" as int) as id,
cast("customer_id" as int) as customer_id,
cast("zipcode" as varchar) as zipcode,
cast("neighborhood" as varchar) as neighborhood,
cast("number" as varchar) as address_number,
cast("additional_address" as varchar) as additional_address,
cast("street" as varchar) as street,
cast("city" as varchar) as city,
cast("state" as varchar) as state,
TRY_CAST(created_at AS timestamp) as customer_address_created
from source
),
final as(
SELECT
id,
customer_id,
zipcode,
neighborhood,
address_number,
additional_address,
street,
city,
state,
customer_address_created
FROM (
SELECT
s0.*,
ROW_NUMBER() OVER
(PARTITION BY
customer_id,
zipcode,
neighborhood,
address_number,
additional_address,
street,
state
ORDER BY
customer_id,
customer_address_created desc
) as row_num
FROM step_0 s0
)
WHERE row_num = 1
)
select * from final
This query is transformed into a new tabled named silver_customer_address. At first, the transformation goes well. But when I compare the values of the query above with the silver_customer_address generated table I find different results. For example, the summing of the id column in both tables generates the following:
- 1734316709196 (the id sum resulted from the above query)
- 1734335121317 (the id sum resulted from the silver_customer_address)
I'm assuming the silver_customer_address must have the same results as the query used in DBT transformation (the one showed above). Why, then, the results are different? I suppose the issue is related with the window function, but don't know why.