i've kept a very basic conf file that reads logs from a folder and pushing it to kafka like so:
input {
file
{
add_field => { "app_id" => "12345"}
path => "/tmp/logs/*.log"
start_position => "beginning"
}
}
output {
if [app_id] == "12345" {
kafka {
topic_id => "mytopic"
bootstrap_servers => "kafka_topic_id:9043"
ssl => false
}
}
}
My logstash logs show "pipeline main started". However I do not see anything on kibana. This means either stream is not being pushed to kafka or else from kafka to elastic search data is not being pushed. Have I missed something in the conf?