Prometheus alert configurations not working as expected

Question

I am unable to create Prometheus alerts based on evaluating an expression, and the problem I have is as below.

The following expression works, and alerts if the count is more than 100, and evaluates the expression to the query_result_employees_count say 110

query_result_employees_count > 100

But I would like to do this check, based on the query_result_total_registered as below:

query_result_employees_count > query_result_total_registered

This always gives me an empty result, although query_result_total_registered is 100 or lesser.

How do I get this expression to work, so that the alerts will be triggered based on both the counts?

Marcelo Ávila de Oliveira · Answer

If the "query_result_employees_count" and "query_result_total_registered" queries have different labels, the comparison will return an empty list.

Try to use the aggregation operators (ex: sum, avg, etc) with the "by" or "without" clause to include or exclude labels.

See more info about comparison binary operators in Prometheus documentation here, and about aggregation operators here.