DBT transformation generating different results

Viewed 19

I have the following query using by DBT to transform one of my datasets.

with source as (
    select * from {{source("datalake", "customer_address")}}
),

step_0 as (
    select        
        cast("id"                      as int)          as  id,                                    
        cast("customer_id"             as int)          as  customer_id,                                   
        cast("zipcode"                 as varchar)      as  zipcode,                                    
        cast("neighborhood"            as varchar)      as  neighborhood,                                  
        cast("number"                  as varchar)      as  address_number,                                    
        cast("additional_address"      as varchar)      as  additional_address,                                   
        cast("street"                  as varchar)      as  street,                                   
        cast("city"                    as varchar)      as  city,                                   
        cast("state"                   as varchar)      as  state,                                   
        TRY_CAST(created_at            AS timestamp)    as  customer_address_created

    from source 
    
),

final as(

    SELECT
        id,                                    
        customer_id,                                   
        zipcode,                                    
        neighborhood,                                  
        address_number,                                    
        additional_address,                                   
        street,                                   
        city,                                   
        state,                                   
        customer_address_created
    FROM (
        SELECT 
            s0.*,
            ROW_NUMBER() OVER 
                (PARTITION BY
                    customer_id, 
                    zipcode, 
                    neighborhood, 
                    address_number, 
                    additional_address, 
                    street,
                    state
                 ORDER BY 
                    customer_id,
                    customer_address_created desc
                ) as row_num
        FROM step_0 s0
    )
    WHERE row_num = 1
    )

select * from final

This query is transformed into a new tabled named silver_customer_address. At first, the transformation goes well. But when I compare the values of the query above with the silver_customer_address generated table I find different results. For example, the summing of the id column in both tables generates the following:

  • 1734316709196 (the id sum resulted from the above query)
  • 1734335121317 (the id sum resulted from the silver_customer_address)

I'm assuming the silver_customer_address must have the same results as the query used in DBT transformation (the one showed above). Why, then, the results are different? I suppose the issue is related with the window function, but don't know why.

0 Answers
Related