Looking for example how to monitor Container Memory Usage with Prometheus.
It reports all the containers ok if we using this query:
(container_memory_usage_bytes / container_spec_memory_limit_bytes) * 100 > 90
However works ok if container does not have a memory limit defined. The the divisor is 0, and the results are +Inf, meaning that the alert triggers incorrectly since +Inf matches > 90.
Any suggestions how to properly use Container Memory Usage monitoring?
I asked the same question from a different perspective just a few days earlier here. So far I have not found an answer. I have settled with adding another label has_memory_limit that I use to only alert on containers that have a limit defined.
Okay, I have figured it out:
((container_memory_usage_bytes / container_spec_memory_limit_bytes) != +Inf) * 100 > 52
Since positive infinity, negative infinity and NaN are numbers in Prometheus, you can simply filter them out with != +Inf.