Slow subquery: group by a groupwise maximum

Viewed 92

I have two tables:

CREATE TABLE share_prices (
    price_id int(10) unsigned NOT NULL AUTO_INCREMENT,
    price_date date NOT NULL,
    company_id int(10) NOT NULL,
    high decimal(20,2) DEFAULT NULL,
    low decimal(20,2) DEFAULT NULL,
    close decimal(20,2) DEFAULT NULL,
    PRIMARY KEY (price_id),
    UNIQUE KEY price_date (price_date,company_id),
    KEY company_id (company_id),
    KEY price_date_2 (price_date)
) ENGINE=InnoDB AUTO_INCREMENT=368586 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

And

CREATE TABLE rating_lookup (
    rating_id int(11) NOT NULL,
    start_date date DEFAULT NULL,
    start_price decimal(10,2) DEFAULT NULL,
    broker_id int(11) DEFAULT NULL,
    company_id int(11) DEFAULT NULL,
    end_date date DEFAULT NULL,
    PRIMARY KEY (rating_id),
    KEY idx_rating_lookup_company_id (company_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

This is the current query:

SELECT broker_id, count(rating_id)

FROM (

    SELECT rating_lookup.*,
    share_prices.company_id as correct_company,
    share_prices.price_date,
    max(high) as peak_gain,
    ( ( ( max(high) - rating_lookup.start_price ) / rating_lookup.start_price ) * 100 ) as percent_gain

    FROM rating_lookup, share_prices

    WHERE share_prices.price_date > rating_lookup.start_date 
    AND share_prices.price_date < ifnull(end_date, curdate())
    AND share_prices.company_id = rating_lookup.company_id

    GROUP BY rating_id

    HAVING percent_gain > 5

) correct

GROUP BY broker_id

Currently this query takes 10.969 sec.

The isolated subquery takes 0.391 sec (duration) / 10.438 sec (fetch)

Query objective:

Get the total amount of correct ratings per broker_id.

A correct rating is defined as a rating that has a reached + 5% since its start_price.


I am looking to drastically decrease the query time, even if restructuring the database is the only way.


Appendix

Explain of above query:

+---+---------+---------------+-------+--------------------------------------+------------+---+----------------------------------------+---------+---------------------------------+
| 1 | PRIMARY | <derived2>    | ALL   |                                      |            |   |                                        | 3894800 | Using temporary; Using filesort |
| 2 | DERIVED | rating_lookup | index | PRIMARY,idx_rating_lookup_company_id | PRIMARY    | 4 |                                        |   18200 | Using where                     |
| 2 | DERIVED | share_prices  | ref   | price_date,company_id,price_date_2   | company_id | 4 | brokermetrics.rating_lookup.company_id |     214 | Using where                     |
+---+---------+---------------+-------+--------------------------------------+------------+---+----------------------------------------+---------+---------------------------------+

share_prices ~ 375,000 rows

rating_lookup ~ 18,000 rows with around 46 unique brokers

3 Answers
Related