Logstash S3 output prefix - event date field value

Forece85 0
Viewed 17

How to set Logstash S3 output prefix dynamically with an event field value in format: "%{+YYYY}/%{+MM}/%{+dd}/%{+HH}" ?

input:
{"record_time":"2017-03-09T04:07:51.520Z"}

required s3 prefix: 
2017/03/09/04
logstash-configuration elk logstash
1 Answers

You can use grok to match record_time to extract year, month, day, hour and then mutate into s3 prefix:

grok {
  match => {
    "record_time" => "%{INT:year}-%{INT:month}-%{INT:day}T%{INT:hour}:%{GREEDYDATA}"
  }
}

mutate {
  # Create s3 prefix
  add_field => {
    "s3_prefix" => "%{year}/%{month}/%{day}/%{hour}"
  }

  # If you don't need separate values, remove them
  remove_field => ["year", "month", "day", "hour"]
}
nenadp0
Related