We've adopted centralized logging using the ELK stack. However, I'm at a loss on how to group messages based on substring (or regex or more complex processing) of their content.
Example:
- Message 1: deleted object id [123]
- Message 2: merged object id [123] to [456]
- Message 3: failed to merged object id [123] to [789]
and so on.
I would like to be able to group-by on messages answering a specific filter (e.g. "failed to merge") but based on a regex (e.g. the first object id - "object id [(.*?)]" or something like that), so I would get a count of how many times object X failed to merge to any other object.
Is this possible with Kibana/ES/Lucene? Ideally I would like to do this through the Kibana UI, but I can almost just as easily use the query language. What I can't do is define calculated/scripted fields for each such query.