I want to move my logging for AWS Lambda from CloudWatch to Elk, using ECS Task or Vm.All my Lambda functions are in Python.To clarify, I want to get rid of CloudWatch, with no streaming to it at all. Question: How do i make my AWS lambda to push logs to AWS ECS Task or Vm instead of CloudWatch logs?
1 Answers
AWS Lambda is serverless and hence it is not easy to get logs of AWS Lmabda directly and Lambda writes the logs to Cloudwatch. Instead of removing logs from Cloudwatch, it is better to stream the logs from Cloudwatch to ELK setup by using an additional Lambda function.
CloudWatch Logs has a feature called Subscriptions for “Real-time Processing of Log Data” which allows sending log events to a Kinesis Stream, Kinesis Firehose Stream, or a Lambda. It is created on a log group. Each log group contains a number of log streams. For each of our existing Lambdas’ log groups, we create a subscription to send the Logs to the new Lambda which will send the logs to the logstash server.
You can refer to this article which explains it in much detail.
Related