I have a spark streaming job that loads the data in apache hudi table every 10 seconds. It update the row in hudi table if the row already exists. Actually, it is doing an upsert operation.
But in hudi table, there is an amount column that is also updated with the new value.
for example
1 batch, id=1, amount value=10. --> in table, amount value = 10
2 batch, id=1, amount value=20. --> in table, amount value = 20
But I need the amount value should 30 not 20. I need to incrementally aggregate the amount column.
Does hudi support incremental aggregation usecase without using external caching/db?