MYSQL - get three or more consecutive rows by time (on same date) where amount is minimum

Viewed 81

I have a database with the following columns. I have added some sample data to show formatting.
date, time, amount
2021-10-14, 13:00, 15.40
2021-10-14, 13:01, 9.34
2021-10-14, 13:02, 10.12
2021-10-14, 13:03, 7.44

There are 2.6 million rows in the database spanning two years. Each row is an increment of 1 minute.

I need to write sql that will output and group rows that are continuous by minute for the same date, where the amount is greater than 8.00 and there are a minimum of 3 consecutive rows.

This would then find an example like:
2021-11-30, 14:44, 8.04
2021-11-30, 14:45, 9.41
2021-11-30, 14:46, 9.27
2021-11-30, 14:47, 10.54
2021-11-30, 14:48, 11.09
2022-03-13, 08:22, 36.44
2022-03-13, 08:23, 17.38
2022-03-13, 08:24, 11.86

2 Answers

So if I understand correctly you only want to select the rows that are part of a 3 minute (minimum) consecutive sequence where amount >= 8 ?

I'm not sure about the performance but this seems to work:

Setup:

CREATE TABLE series
(
    id       INT  NOT NULL PRIMARY KEY AUTO_INCREMENT,
    date     DATE NOT NULL,
    time     TIME NOT NULL,
    datetime DATETIME GENERATED ALWAYS AS (TIMESTAMP(date, time)),
    amount   decimal(5, 2),
    INDEX (amount)
);

INSERT INTO series (date, time, amount)
VALUES ('2021-11-30', '14:40', 7),

       ('2021-11-30', '14:41', 8),
       ('2021-11-30', '14:42', 8),
       ('2021-11-30', '14:43', 8),
       ('2021-11-30', '14:44', 8),

       ('2021-11-30', '14:45', 7),

       ('2021-11-30', '14:46', 8),
       ('2021-11-30', '14:47', 8),
       ('2021-11-30', '14:48', 8),

       ('2021-11-30', '14:49', 7),

       ('2021-11-30', '14:50', 8),
       ('2021-11-30', '14:51', 8),

       ('2021-11-30', '14:52', 7)
;

The solution:

SELECT date, time, T.amount
FROM (SELECT date,
             time,
             datetime,
             amount,
             LAG(datetime, 2) OVER (order by datetime)  AS tmin2,
             LAG(datetime, 1) OVER (order by datetime)  AS tmin1,
             LEAD(datetime, 1) OVER (order by datetime) AS tplus1,
             LEAD(datetime, 2) OVER (order by datetime) AS tplus2
      FROM series
      WHERE amount >= 8) T
WHERE TIME_TO_SEC(TIMEDIFF(T.datetime, T.tmin2)) = 120
   OR TIME_TO_SEC(TIMEDIFF(T.datetime, T.tplus2)) = -120
   OR (TIME_TO_SEC(TIMEDIFF(T.datetime, T.tmin1)) = 60 AND TIME_TO_SEC(TIMEDIFF(T.datetime, T.tplus1)) = -60)
ORDER BY datetime;

Explanation:

  • First we filter out the values < 8 using a WHERE-statement.
  • Then we peek into the previous two and next two rows ordered by datetime to see if the current to is part of a 3 min sequence and filter based on that criteria.

Here is my solution:

Table Definition : 
CREATE TABLE YourTbl (
  date date DEFAULT NULL,
  time time DEFAULT NULL,
  amount decimal(4,2) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;

INSERT INTO YourTbl VALUES
('2021-10-12','16:30', 20.40),
('2021-10-12','14:21', 19.34),
('2021-10-14','13:00', 15.40),
('2021-10-14','13:01', 9.34),
('2021-10-14','13:02', 10.12),
('2021-10-14','13:03', 7.44),
('2021-11-30', '14:44', 8.04),
('2021-11-30', '14:45', 9.41),
('2021-11-30', '14:46', 9.27),
('2021-11-30', '14:47', 10.54),
('2021-11-30', '14:48', 11.09),
('2022-03-13', '08:22', 36.44),
('2022-03-13', '08:23', 17.38),
('2022-03-13', '08:24', 11.86);

Lets test the query:

SELECT 
    date,time,amount FROM 
    (SELECT date,time,amount,
    LEAD(minute(time),1) OVER(PARTITION BY date ORDER BY time) as leadtime,
    LAG(minute(time),1) OVER(PARTITION BY date ORDER BY time) as lagtime,
    (minute(time) - LAG(minute(time),1) OVER(PARTITION BY date ORDER BY time)) as minute_forward_difference,
    (LEAD(minute(time),1) OVER(PARTITION BY date ORDER BY time) - minute(time)) as minute_backward_difference
    FROM YourTbl
    WHERE amount > 8.0
    ) as tblder
    WHERE (minute_forward_difference = 1) OR (minute_backward_difference = 1)
    GROUP BY date,time,amount;

Resultset:

Click here to see image.

Related