Our API (.Net Core) running in AWS EC2 creates multi-line log entries like this:
2021-10-21 15:02:22.968|Error|{
"Message": "An error has occurred."
}|Product.Api.Exceptions.ApiException: {
"Message": "An error has occurred."
}
at Product.Api.Utils.Http.ResponseValidationHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in C:\BuildAgent\work\Product.Api\Utils\Http\Response
ValidationHandler.cs:line 26
These are fed directly into AWS Cloudwatch but are split into multiple log entries, like this:
2021-10-21T15:02:25.123+01:00 2021-10-21 15:02:22.968|Error|{ "Message": "An error has occurred."
2021-10-21T15:02:25.123+01:00 }|Product.Api.Exceptions.ApiException: { "Message": "An error has occurred."
2021-10-21T15:02:25.123+01:00 } at Product.Api.Utils.Http.ResponseValidationHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in C:\BuildAgent\work\Product.Api\Utils\Http\Response
We then send them directly to DataDog, so we need them correct in Cloudwatch before we do.
If the multiline log was a single valid JSON string I believe it would parse correctly, but because there is a combination of string, JSON and pipes(|) it seems to just split on what it thinks are the separate JSON objects.
Is there anyway to keep it as one log entry in Cloudwatch?
I've seen a lot of solutions, but they all seem to relate to Cloudwatch Insights/Metrics, but we send these logs straight to DataDog, so don't analyse them in Cloudwatch itself.