Grafana RAG Status different thresholds for Amber and Green from Prometheus Datasource

user_1568538 0
Viewed 109

I am using a "singlestat" Grafana visualisation to create a RAG status from the following query:

sum(up{container_name="my-application",environment_name="$env"})

enter image description here

So this will tally up the number of running containers in say the production environment. The following visualisation gives a RAG status panel that is RED on zero containers found, AMBER on one container and GREEN on two or more.

Grafana Query Visualisation

So far so good but in other environments like development the number of containers running for "my-application" is only one so RAG status is always reported as AMBER.

How do I handle this? I can't see a way of configuring the "singlestat" visualisation so should I manipulate the returned value from the query and if so how?

grafana prometheus promql
2 Answers

Try using avg(up{container_name="my-application",environment_name="$env"}). If it returns 1, then all the containers are live. If it returns value lower than 1, then something is wrong. The returned value can be multiplied by 100 and interpreted as the percentage of healthy containers.

user_2749370

The answer to this was to modify the query and falsely inflate the returned value for environments that are not running the production minimum of two containers.

(
   sum(up{container_name="my-container", environment_name=~"dev|integration", environment_name="$env"}) + 1 
   AND
   (sum(up{container_name="my-container", environment_name=~"dev|integration", environment_name="$env"} == 1)
)
OR
sum(up{container_name="my-container", environment_name=~"dev|integration", environment_name="$env"}) != 1
OR
sum(up{container_name="my-container", environment_name="prod", environment_name="$env"})
OR
on() vector(0)

Answer from here: PromQL if then statement equivalent

user_15685380
Related