Kibana - get logs surrounding certain document with patterns to mark start and end range of a context

user_4537090 0
Viewed 769

For example, I see an error log in Kibana, but I am not interested only in this error, but also the context of this line, i.e., I want to know what happens before and after this error. Such as:

  • the order failed with status "FAILED", but the log just before this line would contain the method name who caused this error
  • some 5-10 lines before this, I know there would be a line like "Start processing order xxxxx with status xxx"
  • and 15-20 lines after this log, there would be something like "End processing with status xxx"

All this together, marks a life cycle of processing of this particular order. And all these lines are what I mean by saying "context".

How can I get all these lines as a search in Kibana?(Let's suppose all the literals are in the field "message")

For now, I know we can "view surrounding documents", but that is not efficient enough.

kibana elasticsearch
1 Answers

https://www.elastic.co/guide/en/kibana/current/xpack-apm.html

Well, just learn about Elastic APM and it can solve part of the problem. APM can record "span" and "transactions" to form "distributed trace", then add info to the field "trace" to logs and then we can aggregate all logs with same trace id to learn the context of this event across the microservices.

The question now changes to "How to use APM to add trace". And, one of our microservices is reactive, which cannot be easily adapted to use APM: Reactive pipeline context is thread-based, between the threads there are no easy way to transfer the trace from one context to another. So this is the part APM cannot solve.

But at least now we know in imperative apps we have a way.

user_45370900
Related