AWS CloudWatch Logs metrics broken graph

user_7635030 0
Viewed 2432

We've configured a metric filter for a CloudWatch Log Group which collects data for one of our processes.

Initially, we had only one ec2 instance associated with the log stream of the log group. The retrieved data from the logs was showing up nicely in the graph.

However, we want to have multiple instances connected to that same log stream or log group. When we configured another instance for the log group, the graph becomes broken:

Broken-Graph

From what we observed, it looks like the points are only connected if they are consecutive data derived from the latest instance. Otherwise, it just shows a point.

All the data shown are still correct. It's just that they aren't all connected. Is there a way to have all the points connect?

amazon-cloudwatch-metrics amazon-cloudwatch amazon-cloudwatchlogs amazon-web-services
3 Answers

This might be a recently-added feature, but there actually is a way to force connections between discontinuous data points in a CloudWatch graph: you can use metric math. There is a 'FILL' expression which fills gaps in a time series.

You can add a metric math expression with the 'Math expression' button in the 'edit graph' UI:

cloudwatch_gaps

The red lines represent some intermittent data points, and the 'FILL' expression gives you three options for how to fill in the gaps:

  • A scalar will fill the gaps with a constant value (blue line, gaps set to 0).

  • LINEAR will perform a linear interpolation between the two points on either side of a gap (orange line).

  • REPEAT will repeat the last received data point until a new one arrives (green line).

Once you're happy with the gap-filling behavior, you can un-check the original metric's row in the table to hide it when the graph is displayed.

user_17144290

For the benefit of others who may come across this question, in the context of training deep neural networks, I have found that changing the Period value to a larger value caused the points to connect (I changed it from 5 Seconds to 15 Minutes).

user_50957780
Related