How to remove particular lines from input file if starts with special character in logstash

user_10885574 0
Viewed 265

I am trying to remove all the lines starting from < in my input file in logstash. I tried below filter but no success.

I am new to ELS stack, please pardon if it has already answered.

Sample Input lines

Thread #1: t@-1680123584, lwp=22843, ref=0x1b0f8550, session=9F0071A66D89544155D149CCE2453E9A:mx2135649930e123d964:(WebServiceFacade.java:84), ms=0x78714bc0
<Start Stack Trace>
<1 - ADK Verbose Trace Entry>

  <2 - Launching Program Object>
  
  <3 - ADK Verbose Trace Entry>

  <4 - Launching Program Object>

<End Stack Trace>
Thread #2: t@-1680123584, lwp=22843, ref=0x1b0f8550, session=9F0071A66D89544155D149CCE2453E9A:mx2135649930e123d964:(WebServiceFacade.java:84), ms=0x78714bc0
<Start Stack Trace>
  <1 - ADK Verbose Trace Entry>

  <2 - Launching Program Object>
  
  <3 - ADK Verbose Trace Entry>

  <4 - Launching Program Object>

<End Stack Trace>

My current filter is like

    if[message =~ "^<.*"] {
        drop { }
    }

But somehow it is not working, I am still getting these lines in my output

 "message" => "Thread #1: t@-1680123584, lwp=22843, ref=0x1b0f8550, session=9F0071A66D89544155D149CCE2453E9A:mx2135649930e123d964:(WebServiceFacade.java:84), ms=0x78714bc0\r\n<Start Stack Trace>\r\n<1 - ADK Verbose Trace Entry>\r\n\r\n  <2 - Launching Program Object>\r\n  \r\n  <3 - ADK Verbose Trace Entry>\r\n\r\n  <4 - Launching Program Object>\r\n\r\n<End Stack Trace>\r",
logstash-grok elk logstash elasticsearch
2 Answers

You should have square brackets around the field name. Parentheses around the expression are optional. Try

if [message] =~ "^<" { drop { } }
user_117929770

I managed to fix it, I am using multiline codec in input, because of that it is not considering the lines with \r or \n. My actual expression is working fine if I remove multiline codec.

if[message =~ "^<.*"] {
        drop { }
    }
user_108855740
Related