CloudWatch Monitoring and Notifications

user_10393664 0
Viewed 409

I am using various AWS services (Lambda, Glue, S3, Redshift, EC2) for ETL processing. I am trying to create a 'log ETL flow' to have monitoring and notifications sent out (email or otherwise) when a step fails in the process.

I have checked that each service I am using has metrics being logged in CloudWatch. I am now trying to figure out a smart way of processing this data in order to send out notifications when a step fails and/or have a central monitoring of the entire flow.

Are there any best practices or examples of this setup?

amazon-cloudwatch amazon-cloudwatchlogs amazon-web-services
1 Answers

It seems to be the perfect case for the CloudWatch Alarms.

You can create a CloudWatch alarm that watches a single CloudWatch metric or the result of a math expression based on CloudWatch metrics. The alarm performs one or more actions based on the value of the metric or expression relative to a threshold over a number of time periods. The action can be an Amazon EC2 action, an Amazon EC2 Auto Scaling action, or a notification sent to an Amazon SNS topic.

You can create a chain CloudWatch Alarm -> SNS

You can either use SNS to notify users via SMS or Push Notifications.
Or you can do one step more SNS -> SES to deliver emails.

user_7888330
Related