Logstash beats input "invalid version of beats protocol"

user_10512384 0
Viewed 12090

I'm writing a kibana plugin and a logstash pipeline. For my tests, I just wrote a logstash input like that:

input {

   beats {
        port => 9600
        ssl => false
        ssl_verify_mode => "none"
    }

}

But when I try to open a connection with node (code above):

invoke = (parameters, id, port, host) => {
        var fs = require('fs');

        console.log(`Sending message in beats, host= ${host}, port= ${port}, message= ${parameters.message}`);

        var connectionOptions = {
            host: host,
            port: port
        };

        var client = lumberjack.client(connectionOptions, {rejectUnauthorized: false, maxQueueSize: 500});

        client.writeDataFrame({"line": id + " " + parameters.message});
    }

logstash gives to me "invalid version of beats protocol: 22" and "invalid version of beats protocol: 3":

Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.11.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.11.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        ... 9 more
[2020-08-11T07:49:47,954][INFO ][org.logstash.beats.BeatsHandler] [local: 172.22.0.40:9600, remote: 172.22.0.1:33766] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
[2020-08-11T07:49:47,955][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$300(AbstractChannelHandlerContext.java:61) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:253) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.49.Final.jar:4.1.49.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.11.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.11.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
        ... 11 more
elastic-beats elk logstash kibana node.js
2 Answers
  1. Instead of use beats input you could try to use tcp input.

Example:

input {
  tcp {
    port => "9600"
    codec => "json"
  }
}
  1. If you are using beats input and you want to use Logstash to perform additional processing on the data collected by Filebeat, you need to configure Filebeat to use Logstash.

To do this, you edit the Filebeat configuration file to disable the Elasticsearch output by commenting it out and enable the Logstash output by uncommenting the Logstash section:

output.logstash:
     hosts: ["127.0.0.1:5044"]

You can read more on https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html

user_146532920

For understanding more about this error we will need to see the filbeat.yml. Input plugin that you have used is a valid input plugin but in filbert.yml you might have not had output.logstash value or probably made some other mistakes. Can you please check if you have sent the output to Elasticsearch or Logstash ?

Please ensure you have this line of code in your filbert.yml

output.logstash:
  hosts: ["127.0.0.1:5044"]
user_171280410
Related