I have a jdbc input which get the user records (millions of result) from table1 and in filter i am using a jdbc_streaming because based on user's visited products (which is 1 level array ex - [[123], [5,4,7]]) I need to fetch some data from table2 (millions of records).
First Approach - I was trying to fetch the data from table2 via "IN" clause passing those ids in array using prepared statement but i think jdbc_streaming prepared statement do not support that.
Second Approach - Get all the data from table2 for each event and manually map it, it take too long time and for each event i have to run a query.
Is there any way i can cache those results from table2, so it does not query throughout he pipeline?