Prometheus: coalesce values for missing labels in binary operations

user_1003190 0
Viewed 1173

I have two metrics with matching labels, both counters:

accounts_created_total{provider="auth0"} 738
accounts_created_total{provider="google} 980

accounts_deleted_total{provider="auth0"} 65

I'd like to calculate the number of existing accounts from those two metrics. I came up with this:

accounts_created_total - accounts_deleted_total

# which results in

{provider="auth0"} 673

# Note the missing provider="Google"

Unfortunately, there's no account_deleted_total for provider="Google", and so I only get the result for provider="auth0".

Is there a way I can tell prometheus to "make up" the missing labels? That would typically be equivalent to a coalesce in SQL.

prometheus monitoring
1 Answers

You can complete a time serie using the OR binary operator:

vector1 or vector2 results in a vector that contains all original elements (label sets + values) of vector1 and additionally all elements of vector2 which do not have matching label sets in vector1.

Assuming you want to default accounts_deleted_total to 0, the following expression uses accounts_created_total as second vector to extract the labels and multiplying by 0 ensures the value is reset:

accounts_deleted_total OR (accounts_created_total * 0)

In the case of autho0, the label exists in accounts_deleted_total and the second part will not be used ; to the contrary, for google, second part will yield

{provider="google"} 0

Finally, you can use it in your expression:

accounts_created_total - (accounts_deleted_total OR (accounts_created_total * 0))

In your specific case, since you are using the same metric to extract the labels, you can even simplify the expression to:

(accounts_created_total - accounts_deleted_total) OR accounts_created_total
user_34808080
Related