Calculate success rate with prometheus when the numerator is null

user_7437860 0
Viewed 3461

We have many use-cases when we want to calculate success rate but there were not tasks that succeeded, we would expect that the success rate will be 0, but it's none.

lets take these metrics as an example:

http_requests_total{state="failure"} = 10

http_requests_total{state="succes"} - never happened so there is no value at all

I would like to calculate the success rate (which is 0 in this case). Performing division of:

http_requests_total{state="success} / sum(http_requests_total) ignoring (state)

return no data points.

Any suggestion? Am I doing it wrong?

Tried to perform:

(http_requests_total{state="success} or 0)/ sum(http_requests_total) ignoring (state)

but or operator doesn't work

I expect the output will be 0, but it's "No data points"

prometheus
2 Answers

This is not that simple to do, which is one of the reasons why it is recommended not to have metrics which only appear when something happens.

If you cannot handle no data as zero success rate at a later stage (eg. in Grafana by using the "null as zero" option) a version of or which seems to have all the appropriate labels matching correctly could be:

http_requests_total{state="success"} or
 label_replace(0 * sum without (state) (http_requests_total), "state", "success", "__name__", ".*")

This (ab)uses label_replace to synthesize elements with state="success" where there are none and does not look pretty.

user_12360450

The following PromQL query should return zero success rate even if http_requests_total{state="success"} is missing:

(sum(http_requests_total{state="success"} > 0) or vector(0))
  /
sum(http_requests_total > 0)

Unfortunately Prometheus doesn't provide the ability to calculate success rates grouped by additional labels if some samples may be missing :( For example, the following query would return gaps instead of zeroes if some samples for http_requests_total{state="success"} are missing:

sum(http_requests_total{state="success"}) without(state)
  /
sum(http_requests_total) without(state)

This task is easy to solve with VictoriaMetrics (this is Prometheus-like monitoring system I work on) - just add default 0 to the query:

(
  sum(http_requests_total{state="success"}) without(state)
    /
  sum(http_requests_total) without(state)
) default 0

See MetricsQL docs for more details.

user_2749370
Related