Terraform AWS Cloudwatch Metric Filter with Dimensions

Question

I'm trying to configure Terraforms to create a Metric_filter and a Metric_alarm for the filter. When the alarm is triggered it sends an email out via an SNS topic.

I can get it working by configuring AWS manually without using Dimensions. However, as soon as I add Dimensions on the metric_filter it doesn't work - either manually in AWS or in terraforms code.

My understanding is that the dimensions will create a separate count for each variable in the json it specifies. That means if there are 5 logs thrown because of errors in 5 different tables, I should get a separate email for each table.

For example, I can find my log by filtering for the metric_filter pattern "{$.LogTitle = MyDataLog }".

This matches "LogTitle": "MyDataLog" in the json. And it successfully finds the logs.

I want a dimension to create seperate emails for each table I'm monitoring in my API. In the log messages, the TableName can change.

This does not work in the dimensions section:

dimensions = {
  TableName = "$.TableName"
}

Or in the AWS console manually:

In both cases, no data is generated, it's like it can't see the logs or is not recoding them. If I remove this line then everything works, but it's a generalized count for ALL tables not each table individually.

Any help is appreciated? Do I need to use a lambda or something?

Here is one of my logs:

{
 "@t": "2022-09-07T03:54:45",
 "@mt": "{LogTitle} - Table {TableName} is out of date.",
 "@l": "Warning",
 "LogTitle": "MyDataLog",
 "TableName": "TableOne",
 "SourceContext": "MyApp.HomeController.Data.handler",
 "ActionId": "be41erg7-066y-4f8d-abc1-085drsgat4",
 "ActionName": "MyApp.HomeController.Data",
 "RequestId": "12tgfr54sfsrt",
 "RequestPath": "/myapi",
 "ConnectionId": "12tgfr54sfsrt",
 "User-Agent": [
    "Mozilla/5.0 blah"
 ],
 "Referer": [
    "https://mywebsite.com/myapp"
 ],
 "RequestPort": 12345,
 "RequestIpAddress": "blah",
 "User": "myemail@email.com",
 "ThreadId": 123,
 "ApplicationName": "My App"
}

Here is my terraform metric_filter:

resource "aws_cloudwatch_log_metric_filter" "this" {
  name           = "TableErrorLog"
  pattern        = "{$.LogTitle = MyDataLog }"
  log_group_name = aws_cloudwatch_log_group.this.name

  metric_transformation {
    name      = "TableErrorLog"
    namespace = var.app
    value     = "1"
    dimensions = {
      TableName = "$.TableName"
    }
  }
}

Here is my metric_alarm. The SNS topic is created fine and works. I've just hidden some of it in a variable file:

    resource "aws_cloudwatch_metric_alarm" "table_error_log_alarm" {
  alarm_name                = "TableErrorLogAlam"
  comparison_operator       = "GreaterThanOrEqualToThreshold"
  evaluation_periods        = "1"
  metric_name               = "EventCount"
  namespace                 = var.app
  period                    = "43200"
  statistic                 = "Sum"
  threshold                 = "1"
  datapoints_to_alarm       = "1"
  dimensions = {
      TableName = "$.TableName"
    }
  alarm_description         = "This metric monitors Table Error Logs"
  alarm_actions             = ["${module.sns.topic_arn}"]
}

So I'm after a working solution to have a filter trigger an alarm and have a dimension of that alarm be the individual TableName the log error specifies. Not sure I'm doing the dimensions right... Do I maybe need to have a lambda as a dimension or something?

Terraform AWS Cloudwatch Metric Filter with Dimensions

0 Answers