Parsing JSON to Kibana using Grok Filter

Viewed 260

I want to parse a JSON file like this

{
"Format": "IDEA0",
"ID": "2b03eb1f-fc4c-4f67-94e5-31c9fb32dccc",
"DetectTime": "2022-01-31T08:16:12.600470+07:00",
"EventTime": "2022-01-31T01:23:01.637438+00:00",
"Category": ['Intrusion.Botnet'],
"Confidence": 0.03,
"Note": "C&C channel, destination IP: 192.168.1.24 port: 8007/tcp score: 0.9324",
"Source": [{'IP4': ['192.168.1.25'], 'Type': ['CC']}]
}

Using this Logstash Conf

input {
        file {
                path => "/home/ubuntu/Downloads/StratosphereLinuxIPS/output/*.json"
                start_position => "beginning"
                sincedb_path => "/dev/null"
        }
}

filter {
        json {
                source => "message"
        }
}

output {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "test-test"
                user => "***"
                password => "***"
        }
        stdout{}
}

But i can't get a proper output in Kibana, the JSON is not separated correctly

Kibana Output

Is there anyone who can help me? any help will i appreciate

Thank You ! Adabi

1 Answers

The problem is not the JSON filter, but your input plugin: file has a delimiter parameter which defaults to \n, so one line in your file will produce one document. You can change the delimiter (maybe to },\n, but depends on how you are writing the files) or use the multiline codec plugin (here's some wisdom) or (even better) write a single JSON on a single line.

Related