Identification/labeling/Tagging in Prometheus

Viewed 71

I am using Prometheus as a monitoring solution, basically we monitor AWS resources like lambda, Api gateway etc and alerts will generate and go to teams channel, we have 500+ lambda created by multiple teams, In AWS itself we are unable to identify which resource created by which team, now we enforced tags in aws to identify the owner of resource in the form of tags, but how i can find out in Prometheus, (we have defined generic definition file, means for all lambda one definition file will work, Can any one please help me to define, add tags in Prometheus so i can identify which resource is created by which team. I saw we can use labeling but as i mentioned we are using one definition file for all the lambda in env how we can achieve it. sample definition file

  • name: generic_lambda_alerts rules:

    Alert on sum of 'Errors' is greater than '0'

    • alert: generic_lambda_errors_sum annotations: description: Detected Lambda error in function {{ $labels.function_name }} summary: Lambda Errors sum runbook_url: url expr: |- sum(aws_lambda_errors_sum{cluster_name=~"som-.*-eks"} offset 5m) by (cluster_name, function_name) > 0 for: 0m labels: dashboard_url: url severity: critical project: projectname
0 Answers
Related