Understanding increase() and rate() used on http_server_requests_seconds_count with prometheus and Grafana

user_6317008 0
Viewed 1708

I tried to obtains these measurements from prometheus:

  1. increase(http_server_requests_seconds_count{uri="myURI"}[10s])
  2. increase(http_server_requests_seconds_count{uri="myURI"}[30s])
  3. rate(http_server_requests_seconds_count{uri="myURI"}[10s])
  4. rate(http_server_requests_seconds_count{uri="myURI"}[30s])

Then I run a python script where 5 threads are created, each of them hitting this myURI endpoint:

What I see on Grafana is:

enter image description here enter image description here

I received these values:

  1. 0
  2. 6
  3. 0
  4. 0.2

I expected to receive these (but didn't):

  1. 5 (as in the last 10 seconds this endpoint received 5 calls)
  2. 5 (as in the last 30 seconds this endpoint received 5 calls)
  3. 0.5 (the endpoint received 5 calls in 10 seconds 5/10)
  4. 0.167 (the endpoint received 5 calls in 30 seconds 5/30)

Can someone explain with my example the formula behind this function and a way to achieve the metrics/value I expect?

micrometer grafana prometheus spring
1 Answers

Increase measure the increase in your window, and rate is the 'per second' rate.

So if you were to imagine that Prometheus write a database row each second of what the current value is. That helps reason through the metrics calculations.

  1. If the total so far was 0 then the metric increased by 5 in the last 10 seconds, the increase is looking back 10 seconds and finding the value and reporting the increase.

  2. The same as 1. but it is looking back 30 seconds.

  3. The 0.5 comes from looking at the window size (10s), measuring the increase them normalizing it to a 'per second' rate. So the increase of 5/10 seconds: .5 calls per second

  4. With the larger window, the 'per second' is spread over the larger window.

increase is easier to reason about, but rate standardizes on the 'per second' unit.

One problem with the 'per second' unit though is that it can be too small to talk through, but it makes it easy to multiple by 60 (to show a per minute) or 3600 (to show a per hour). The window is only used to calculate the window for how far to gather data for the calculation.

user_9703080
Related