The log file that I am using has logs in the following format:
2021-07-29 14:51:00,668 INFO audit [http-nio-8080-exec-4] {"user": "John", "country": "USA"}
2021-07-29 14:51:00,668 INFO audit [http-nio-8080-exec-4] {"user": "Jack", "country": "CA"}
2021-07-29 14:51:00,668 INFO audit [http-nio-8080-exec-4] {"user": "Josh", "country": "USA"}
I want my document to look like the following (excluding the elasticsearch metadata fields) in Elasticsearch
{
"user": "John",
"country": "USA"
}
How can I store the data in above format on Elasticsearch using FileBeat?
Option 1:
I have tried the following options in Filebeat. But it throws error as "error in decoding..."
json.keys_under_root: true
json.add_error_key: true
Option 2:
I haven't disabled the above json options in Filebeat. But log entry stored as single value in message key as below
{
"message": "2021-07-29 14:51:00,668 INFO audit [http-nio-8080-exec-4] {"user": "John", "country": "USA"}"
}