Defining priority for each read query: making some queries run in background

user_3979116 0
Viewed 426

My problem is the following: I run a clickhouse cluster for a low latency web UI doing classical OLAP queries like SELECT ... WHERE... GROUP BY ...

Most of these queries are very fast from 10 ms to 1s and this is perfect.

Yet I also want to use clickhouse for heavier queries for example using string dimensions (URL for example) or slower functionalities like SELECT DISTINCT(COUNT(...)), or simply running on a very large perimeter (all rows).

My point is not to make these queries faster but to avoid that they slow down the other queries by using the full resources of the cluster.

I can imagine a basic scheduling method like this:

  • start your heavy query with full resources
  • any time a light query comes in, suspend the heavy one, and run the light one to completion, then continue the heavy one with full power

Or any other smoother priority method.

Is there something in clickhouse capable of defining that sort of priorities?

clickhouse
1 Answers

There is such feature (unfortunately not documented yet):

:) select * from system.settings where name like '%priority%';

SELECT *
FROM system.settings
WHERE name LIKE '%priority%'

┌─name─────┬─value─┬─changed─┬─description───────────────────────────────────────────────────────────────────────────────────────┐
│ priority │ 0     │       0 │ Priority of the query. 1 - the highest, higher value - lower priority; 0 - do not use priorities. │
└──────────┴───────┴─────────┴───────────────────────────────────────────────────────────────────────────────────────────────────┘

1 rows in set. Elapsed: 0.002 sec.
user_10321170
Related