How to remove no longer valid Gauges

Viewed 4674

I use the Prometheus Java Client to export session information of my application. We want to show how long sessions have been idle.

The problem is that we have a maximum of 1000 sessions and sessions are removed after a certain period. Unfortunately they do not disappear from Prometheus:

Current and expired sessions

My code looks like this:

static final Gauge sessionInactivity = Gauge.build()
    .name("sessions_inactivity_duration")
    .labelNames("internal_key", "external_key", "browser")
    .help("Number of milliseconds a certain session has been inactive")
    .register();

sessionInactivity.labels(internalKey, externalKey, browser).set(inactivityTime);

I tried to do sessionInactivity.clear() during scrapes but obviously this does not empty the content of the Gauge.

3 Answers

The Gauge class has a remove method which has the same signature as the labels method. For your specific example, removing the metrics associated with that gauge would look like this

sessionInactivity.remove(internalKey, externalKey, browser);

The client library documentation states:

Metrics with labels SHOULD support a remove() method with the same signature as labels() that will remove a Child from the metric no longer exporting it, and a clear() method that removes all Children from the metric. These invalidate caching of Children.

Since you are registrating the gauge, you need to remove the gauges which are expired from the registry.

protected MeterRegistry registry;

  ...
  static final Gauge sessionInactivity = Gauge.build()
    .name("sessions_inactivity_duration")
    .labelNames("internal_key", "external_key", "browser")
    .help("Number of milliseconds a certain session has been inactive")
    .register(registry); // use registry

  ...
  // remove expired gauges from registry
  registry.remove(sessionInactivity);
Related