Super noob here so please here me out :
I have a multiline log file written in JSON format and i have successfully written a grok expression to extract variables and values.
Question: this is a JSON endpoint file hence the file always beginning with the 'data' encapsulate the whole log (as one large message). As you can see from the grok link , the expression also deals with the 'data' at the beginning of the file. The rest of the log events are separated by a ',' outside a '}'. How do i deal with this issue as this expression only works for the first event in log? As the file is a large one (over 100mb) in JSON Format, it is a bit difficult to edit it in any text editor as the program will crash as id not be able to remove the 'data' form the beginning of file. Also, I tried to extract values using the exact same expression in a 2 record log, this didnt work . Please can anyone help with this ?enter image description here
{"data":[{"ACTIVITY":"Retrieved Tax Payer information for 10190185-0001","ID":"312285","IR_NUMBER":"27568","NAME":"IBRAHIM BABANGIDA B.","TASK":"Tax Payer information","WHENLOG":"2022-09-01 10:21:23.831219"}
grok :
\{\"data\"\:\[\{\"ACTIVITY\"\:\"%{DATA:activity}\"\,\"ID\":\"%{DATA:logentryID}\"\,\"IR_NUMBER\":\"%{DATA:ir_no}\"\,\"NAME\":\"%{DATA:staff_name}\"\,\"TASK\":\"%{DATA:task}\"\,\"WHENLOG\":\"%{TIMESTAMP_ISO8601:timestamp}\"