Limit number of allowed values for a label whan scraping targets

user_296452 0
Viewed 143

Using prometheus it's nice to keep the number of values a label holds low, to make querying faster.

So, for a metric I'm collecting from several (hundreds) of targets I'd like to have a list of allowed values, all other would end as "other".

I can ofcourse do this on my targets but I would much prefer to have it done centrally at the scraper.

One example could be to have the user agent of web requests as a label but rewrite all of them as "chrome", "firefox", "safari" and "other".

Another to put a client specified header value, "client_name", as a label but make sure that only "oneApp" and "anotherApp" are allowed values, and put all the other ones in "inofficialApp".

This is an example To be clear. This is what I would like for all metrics and from all targets

http_request_duration_seconds_count{client_name="the app"} 1
http_request_duration_seconds_count{client_name="the new app"} 1
http_request_duration_seconds_count{client_name="bogus guy 1"} 1
http_request_duration_seconds_count{client_name="bogus guy 2"} 1

turns into

http_request_duration_seconds_count{client_name="the app"} 1
http_request_duration_seconds_count{client_name="the new app"} 1
http_request_duration_seconds_count{client_name="OTHER"} 2
prometheus
1 Answers

The only way to achieve that with prometheus that I know is to use recording rules:

groups:
  - name: my group
    interval: 30s
    rules:
      - record: my_metric_name:new  # name of new metric
        expr: my_metric_name{instance="foo:9100"}
        labels:
          instance: foo

This is far from ideal because you might have to write a lot of rules like this. Using it don't forget to use some aggregation in expr so that your new metric will have a combined value. Like sum(my_metric{label1=~"some_regex"}) without (label1).

user_113445020
Related