Logstash - Pipeline has terminated

Viewed 2841

I am learning the ELK stack. At this time, I have Elasticsearch and Kibana setup on an AWS instance. I am successfully writing docs to an Elasticsearch index on that index via a Java console app I've written. I can see those logs in Kibana. Now, I want to see those logs input into Logstash on a different machine.

In an attempt to do this, I've installed Logstash. I added the following configuration:

logstash.conf

input { 
  elasticsearch {
    hosts => "https://<aws-instance>.us-east-1.aws.found.io:9243"
    user => "<myUsername>"
    password => "<myPassword>"
    index => "<myIndexName>"
    query => '{ "query": { "match": { "statuscode": 200 } }, "sort": [ "_doc" ] }'
  }
}
output {
  stdout { }
}

When I start Logstash from a console window, I'm expecting to see logs pushed to my Elasticsearch index appear. Instead, I see:

Sending Logstash logs to C:/Temporary/logs which is now configured via log4j2.properties
[...][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[...][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.4.2"}
[...][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[...][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x13bbcec sleep>"}
[...][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[...][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[...][INFO ][logstash.pipeline        ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x13bbcec run>"}

I don't understand this. Why would the "pipeline" terminate? At this point, I don't understand if the issue is with Logstash or connecting to my Elasticsearch index. In either case, I have no idea. I also tried running with the trace log level. Unfortunately, I didn't learn anything from those additional logs. There are no ERROR messages. After the initialization of the Elasticsearch input plugin, I simply see:

[...][DEBUG][logstash.inputs.elasticsearch] Closing {:plugin=>"LogStash::Inputs::Elasticsearch"}

I don't know why it would close though.

0 Answers
Related