I’m looking for a way to transfer cloudwatch logs to s3 bucket. I found I can use subscription filter in cloudwatch. I see two options.
One is CW -> Kinesis Firehose -> S3.
Other one is CW -> lambda -> S3.
Do you know which one is better? I feel like Kinesis firehose is easier to set up, but is using lambda cheaper?
When using lambda you get charged:
For Kinesis you get charged:
Lambda will always be a cheaper solution if its setup correctly.
Kinesis in the back is a lambda.
Sending CloudWatch Logs to S3 using Firehose is way simpler. If you do it using Lambda you will need to handle putting the object on S3 by yourself and have a retrying mechanism if something goes wrong with your Lambda. Lambda is more recommended in cases where you need to process the logs before actually sending them.
It is important to calculate the volume of log data that will be generated beforehand and based on that provision your Firehose accordingly, if you go that way.
Finally, if you don't care about the logs being sent in real-time you can also explore the Export to S3 feature by CloudWatch. You can set up a Lambda that runs periodically and creates an Export for your CloudWatch log group. This is the cheaper solution but with the downside of not transferring your logs in real-time.
