Updating in Loops - Slow performance - Snowflake DB

Viewed 174

I have logic which I am copying over from HANA Stored procedure. There is loop, which iterates for 3k times. Inside, the loop, the tables keeps getting updated. Though each update statement takes only 1-2 seconds, over all the job runs for more than 2 hours. In Hana, the same job gets completed in 5 mins.

What can be done to achieve maximum performance in snowflake? is it possible to change this update to a select statement?

FOR A IN 1 .. :ITERATION_COUNT
DO

/*ITERATIONS IN ORDER TO STEP DOWN THROUGH THE ALLOCATION
 *PROCESS USING THE LOWEST PERCENT TO PLAN. ALLOCATES ONE 
 *CASE SIZE QUANTITY DURING EACH STEP AND ADJUSTS THE
 *VALUES ACCORDINGLY.*/
UPDATE IA_IFC_ALLOCATION.IFC_NEED_ALLOCATION IFC_NEED_ALLOCATION
SET IFC_NEED_ALLOCATION.ALLOCATION_AMOUNT = COALESCE(IFC_NEED_ALLOCATION.ALLOCATION_AMOUNT,0) + IFC_NEED_ALLOCATION.CASE_SIZE,
    IFC_NEED_ALLOCATION.NEW_PROJOH = IFC_NEED_ALLOCATION.NEW_PROJOH + IFC_NEED_ALLOCATION.CASE_SIZE,
    IFC_NEED_ALLOCATION.PERCENT_TO_PLAN  = (IFC_NEED_ALLOCATION.NEW_PROJOH + IFC_NEED_ALLOCATION.CASE_SIZE)/NULLIF(IFC_NEED_ALLOCATION.OUTL,0)
FROM IA_IFC_ALLOCATION.IFC_NEED_ALLOCATION
JOIN (SELECT
    PTP.ITEM,
    MIN(LOC) AS LOC,
    PTP.ZZPOLOC,
    PTP.PERCENT_TO_PLAN
    FROM(
        SELECT 
        ITEM,
        ZZPOLOC,
        MIN(PERCENT_TO_PLAN) AS PERCENT_TO_PLAN
        FROM IA_IFC_ALLOCATION.IFC_NEED_ALLOCATION
        GROUP BY 
        ITEM,
        ZZPOLOC) PTP
    JOIN IA_IFC_ALLOCATION.IFC_NEED_ALLOCATION ON 
    PTP.ITEM=IFC_NEED_ALLOCATION.ITEM
    AND PTP.PERCENT_TO_PLAN =IFC_NEED_ALLOCATION.PERCENT_TO_PLAN 
    GROUP BY 
    PTP.ITEM,
    PTP.ZZPOLOC,
    PTP.PERCENT_TO_PLAN) IFC_NEED_ALLOCATION_MIN_PTP ON 
IFC_NEED_ALLOCATION.ITEM = IFC_NEED_ALLOCATION_MIN_PTP.ITEM
AND IFC_NEED_ALLOCATION.PERCENT_TO_PLAN = IFC_NEED_ALLOCATION_MIN_PTP.PERCENT_TO_PLAN
AND IFC_NEED_ALLOCATION.LOC = IFC_NEED_ALLOCATION_MIN_PTP.LOC
WHERE IFC_NEED_ALLOCATION.RECEIVED_QUANTITY > 0;

UPDATE IA_IFC_ALLOCATION.IFC_NEED_ALLOCATION
SET IFC_NEED_ALLOCATION.RECEIVED_QUANTITY = IFC_NEED_ALLOCATION.RECEIVED_QUANTITY - IFC_NEED_ALLOCATION.CASE_SIZE
FROM IA_IFC_ALLOCATION.IFC_NEED_ALLOCATION
WHERE IFC_NEED_ALLOCATION.RECEIVED_QUANTITY > 0;

END FOR;
1 Answers

I would not "update the table" but instead "build a results set" to push through a single merge operation.

This smells like a recursive CTE.

Related