How to break the streams in Docker awslog driver?

user_14274529 0
Viewed 94

I do have a EC2 instance and a docker container running on it. Currently this docker container uses awslog driver to push logs to CloudWatch. If I go to CloudWatch console, I see a very large log stream (with container id as name) which contains all logs of last 16 days (since I've created the container). It almost seems like if I have this container running for 1 year, this log stream will keep all logs of 1 year. I am not quite sure what is the maximum size limit of a CloudWatch log stream, but most likely it will have a limit, at least I believe.

enter image description here

So my question is;

  1. How to chunk this huge logstream? Hopefully by current date, smth like {{.ContainerId}}{{.CurrentDate}}
  2. What is the maximum size limit of a CloudWatch log stream?
  3. Is it a good practice to append onto a single huge log stream?
amazon-cloudwatch amazon-cloudwatchlogs docker amazon-web-services amazon-ec2
1 Answers

The following is the definition of Cloudwatch Log Stream as defined in the docs, here

Log streams

A log stream is a sequence of log events that share the same source. More specifically, a log stream is generally intended to represent the sequence of events coming from the application instance or resource being monitored. For example, a log stream may be associated with an Apache access log on a specific host. When you no longer need a log stream, you can delete it using the aws logs delete-log-stream command.

Unfortunately what you want is not possible at the moment. Not sure what exactly is your use-case but you can filter the logs streams using time, so separating them is not really necessary. See start-time and end-time in filter-log-events

You might want to define the following awslog driver options to get a better stream name.

awslogs-stream-prefix see docs

user_2541030
Related