SQL check if group is continuous when ordered and return broken groups

Viewed 192

I tried to find a way to list rows, that are breaking continuous groups of records. I say groups, because we could use GROUP BY to list values of groups (but that is not applied, we need particular rows).

Sample data:

CREATE TABLE Test (ID INT, NNO INT, DIDX INT, SIDX INT);

-- Valid sample rows
INSERT INTO Test (ID, NNO, DIDX, SIDX) VALUES
    ( 1,    107     , 7898, 0  ),
    ( 2,    102     , 7883, 0  ),
    ( 3,    53      , 7877, 0  ),
    ( 4,    62      , 7877, 42 ),
    ( 5,    101     , 7870, 81 ),
    ( 6,    103     , 7918, 42 ),
    ( 7,    110     , 7920, 42 ),
    ( 8,    100     , 7919, 0  ),
    ( 9,    24      , 7921, 0  ),
    (10,    85      , 7904, 0  ),
    (11,    85      , 7905, 0  ),
    (12,    85      , 7906, 0  ),
    (13,    85      , 7907, 0  ),
    (14,    85      , 7908, 0  ),
    (15,    85      , 7911, 0  ),
    (16,    112     , 7876, 0  ),
    (17,    5       , 7891, 42 ),
    (18,    80      , 7912, 42 ),
    (19,    66      , 7912, 91 ),
    (20,    22      , 7912, 81 ),
    (21,    60      , 7911, 42 ),
    (22,    60      , 7912, 0  ),
    (23,    78      , 7891, 81 );

-- Disecting row
INSERT INTO Test (ID, NNO, DIDX, SIDX) VALUES 
    (24,    666     , 7906, 120);

EDIT: I probaly mislead some answers a bit by providing an example too much simplified. It then appeared that perhaps the groups could be only broken by a single row. So please add another row into the example data set:

-- Disecting row -2-
INSERT INTO Test (ID, NNO, DIDX, SIDX) VALUES 
    (25,    444     , 7906, 160);

Now if ordered the rows in this particular order:

SELECT ID, NNO, DIDX, SIDX
FROM Test 
ORDER BY DIDX, SIDX;

...the last inserted row will break (disect) group of records, which have NNO=85:

ID          NNO         DIDX        SIDX
----------- ----------- ----------- -----------
...
10          85          7904        0
11          85          7905        0
12          85          7906        0
24          666         7906        120       <<<<<<<<<<<<<<<<<<<
25          444         7906        160       <<<<<<<<<<<< after EDIT <<<<<<<
13          85          7907        0
14          85          7908        0
15          85          7911        0
...

The result should say 85, which is the broken group, or NULL if we would use healthy data without row ID=24.

Another way to look at the problem is, that for each group (even if it contains 1 row), there may not be records of another group which start or end lies between start and end of the queried group. In the provided example, queried group (85) starts with DIDX=7904 and SIDX=0 and ends with DIDX=7911 and SIDX=0 and nothing can fall into the range - which is the case of record ID=24.

I so far tried some approaches like using ROW_NUMBER() OVER (ORDER BY ...), using WITH with MIN and MAX to go through each group and check, if there are rows that fall within (failed so far to construct it) and GROUP BY with MIN and MAX with aim to cross check it with table rows. No attempt is really worth publishing, so far.

Could anyone advice, how I could check continuity of such defined groups?

3 Answers
WITH CTE AS (
SELECT 
  ID
  ,NNO
  ,DIDX
  ,SIDX
  ,LAG(NNO) OVER (ORDER BY DIDX, SIDX) as previousNNO
  ,LEAD(NNO) OVER (ORDER BY DIDX, SIDX) as nextNNO 
FROM Test
) 
SELECT 
 previousNNO as BrokenGroup
FROM CTE
WHERE previousNNO=nextNNO 
AND NNO<>previousNNO 

I used a CTE and WINDOW functions to also keep track of the previous and next group (NNO) for each row. A broken group will be one that has a different current group while the previous and next are the same. From your example with ID 24.

ID          NNO                     DIDX        SIDX
----------- -----------             ----------- -----------
...
12          85  < previous group    7906        0
24          666 < current group     7906        120       <<<<<<<<<<<<<<<<<<<
13          85  < next group        7907        0
...

I'm assuming that any consecutive range of DIDX should only have one NNO. As such there will be no two valid groups that abut each other.

This should help identify the offenders:

with data as (
    select NNO, DIDX, dense_rank() over (order by DIDX) as rn
    from Test
)
select min(DIDX) as range_start, max(DIDX) as range_end
from data
group by DIDX - rn
having count(distinct NNO) > 1;

Getting the actual rows:

with data as (
    select ID, NNO, DIDX, dense_rank() over (order by DIDX) as rn
    from Test
), groups as (
    select DIDX - rn as grp, min(DIDX) as range_start, max(DIDX) as range_end
    from data        
    group by DIDX - rn
    having count(distinct NNO) > 1
), data2 as (
    select *, lead(NNO) over (partition by grp order by DIDX) as next_NNO 
    from Test t inner join groups g
        on t.DIDX between g.range_start and g.range_end
)
select * from data2 where NNO <> next_NNO;

If you're looking for a test to run prior to inserting a row:

with data as (
    select NNO, DIDX, row_number() over (order by DIDX) as rn
    from Test
)
select case when min(DIDX) is not null then 'Fail' else 'Pass' end as InsertTest
from data
group by DIDX - rn
having @proposed_DIDX between min(DIDX) and max(DIDX)
    and @proposed_NNO <> min(NNO);

OK, so inspired by given answers (which I voted up for helping), I came with this code that seems to provide the desired result. I'm not sure though, if it's the cleanest and shortest possible way.

;WITH CTE AS (
    SELECT NNO, 
           DMIN, 
           DMAX,
           SMIN,
           SMAX,
           LEAD(DMIN) OVER (ORDER BY DMIN, SMIN) as nextDMIN,
           LAG(DMAX) OVER (ORDER BY DMIN, SMIN) as prevDMAX,
           LAG(SMAX) OVER (ORDER BY DMIN, SMIN) as prevSMAX,
           LEAD(SMIN) OVER (ORDER BY DMIN, SMIN) as nextSMIN,
           CNT
    FROM (
        SELECT NNO, 
               MIN(DIDX) as DMIN, 
               MAX(DIDX) as DMAX, 
               MIN(SIDX) as SMIN, 
               MAX(SIDX) as SMAX,
               COUNT(NNO) as CNT
        FROM Test
        GROUP BY NNO
        ) as SRC
    )

SELECT * 
FROM CTE
WHERE ((prevDMAX > DMIN OR (prevDMAX = DMIN AND prevSMAX > SMIN)) OR 
       (nextDMIN < DMAX OR (nextDMIN = DMAX AND nextSMIN < SMAX)))
      AND CNT > 1

Perhaps I should give a little explanation. The code finds MIN and MAX border values for each parameter SMIN and DMIN and then find those values for previous and next rows. We also COUNT number of rows in a group.

The conditions in brackets basicaly say, that no record of other group can be in DMIN and DMAX range, and if it's on the borders of the range, then it has to be outside SMIN or SMAX. Finally, a broken group must have more then 1 row; otherwise the query would return not only offendee group, but also first offender.

I should say, that this code has a little flaw and that is a case when there would be an offender with more than one row intact in between ofendee group rows. I should be able to tackle this in post processing, where I have to shuffle rows to achieve intact groups.

Related