Prometheus: 0 offset counter

Viewed 565

Given a Prometheus Counter object with monotonically increasing values, how can I generate a graph where the values begin at 0 for the start of the range in grafana and are grouped by label?

Pseudo formula (how I imagine it. I could be wrong)

foreach(label) plot(events - min_in_window(events)) # within windows

The result should be a monotonically increasing function, where values are offset to 0 at the beginning of the window. 1 event should have an identical increase in the resulting function/graph.

1 Answers

This is fresh from the owen, so perhaps it needs some dressing up. my_metric - max_over_time(((timestamp(my_metric) <= bool (${__from} / 1000)) * my_metric)[$__range:1m])

Original counter plus the same one offset to 0

Deficiency: Won't work for counter resets. For that to work increase() should be used in some place, but wasn't able to figure that out.

TBD:

  • <= bool (${__from} / 1000)) needs polishing. Now it just happens to work, but perhaps there's a better, more robust way to pinpoint the first timestamp.
  • [$__range:1m] needs polishing. The second part, interval, probably depends on scraping interval.

Didn't have time to work on TBDs. If you make it work for you, let us know the polished version.

Related