Adding Alertmanager expression queries while alerting

user_6261078 0
Viewed 705

My intention is to add the query expressions to be displayed in descriptions or summary when alertmanager is alerting through alerting mediums. Is it possible something like this below?

- alert: OutOfDiskSpace
expr: node_filesystem_free_bytes / node_filesystem_size_bytes * 100 < 10
for: 1m
labels:
  severity: Critical
annotations:
  description: "Disk is almost full. The expr query  is {{ $labels.expr }}"
prometheus-alertmanager grafana prometheus promql
1 Answers

I think what you are looking for is $value which will display the value evaluated by the expression. For example consider the following rule

      - alert: service_down
        expr: (probe_http_status_code - 0) != 200
        for: 1m
        labels:
          severity: 3
          threshold: danger
        annotations:
          summary: "HTTP : {{ $value }} for URL *{{ $labels.instance }}*."
          description: "*{{ $labels.instance }}* of job *{{ $labels.job }}* is down"

In the summary, you will get the status code of the API which is down.

In your case the following should work

- alert: OutOfDiskSpace
expr: node_filesystem_free_bytes / node_filesystem_size_bytes * 100 < 10
for: 1m
labels:
  severity: Critical
annotations:
  description: "Disk is almost full. The expr query  is $value"
user_57610110
Related