pandas groupby-transform logic in postgreSQL

Viewed 16

I need to compare a value in one column to all the values in the same column, to search for a pattern obtained from each value among all the others.

For example, in this table, I need to search for the xxx, yyy, zzz paterns from the first Order ID value in all the other Order IDs to find it and create a new column replacing the actual value for the pattern.

Suppliers Order ID
A1 xxx
A1 00xxx
A1 xxx0
A1 200xx
A2 yyy
A2 01yyy0
A2 45yyy
A3 45zzz

Obtaining

Suppliers Order ID Order ID OK
A1 xxx xxx
A1 00xxx xxx
A1 xxx0 xxx
A1 200xx 200xx
A2 yyy yyy
A2 01yyy0 yyy
A2 45yyy yyy
A3 45zzz zzz

I have a problem with the runtime here, because as the table increases, the search space does as well and I end up with an exponential increase in runtime, so I just need to run the search logic *for each supplier (this is where the other column Suppliers needs to play a role), that is, search all the values for Supplier A1 and replace the pattern, then do the same for supplier A2 and so on.

In pandas, that can be achieved using split-apply-combine techinques like df.groupby.transform, but I am totally lost on how to do that using postgreSQL or Vertica SQL.

here what I have tried so far, using a self join on the Order Id columns to create the search space for each value in all the others, but the runtime is not feasible.

WITH cte AS (
   SELECT 
       A."Suppliers" AS "Suppliers", 
       A."Order ID" AS "Order ID", 
       B."Order ID" AS "Order ID OK",
       ROW_NUMBER() OVER (PARTITION BY A."Order ID") AS "DUPLICATE_ORDER_ID"
   FROM table AS A 
       LEFT JOIN table AS B
           ON A."Order ID" LIKE '%' || B."Order ID" || '%'
    )
    SELECT Suppliers, "Order ID", "Order ID OK" FROM cte 
    WHERE "DUPLICATE_ORDER_ID" = 1

Is there anyway I could apply this same code modifiying it to only apply the search logic by supplier and then concatenate the result? (The Order ID values are unique no matter the supplier)

Thank you!!!!!!

0 Answers
Related