Zookeeper for Apache-Kafka: problems with port 2181 in Ubuntu 18.04.01 Server

user_2315094 0
Viewed 10183

I'm trying to figure out how to successfully reach the end of this tutorial, for the installation and first running of Apache-Kafka : How To Install Apache Kafka on Ubuntu 18.04 LTS

After killing the previously running zookeeper instance, I tried, again!, to start the zookeeper server again, but when the process reaches the phase of port binding, it stops forever..: KafkaActivationFailed

marco@PC:/usr/share/zookeeper$ sudo lsof -i :2181
COMMAND  PID      USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    4545 zookeeper   32u  IPv6  43042      0t0  TCP *:2181 (LISTEN)
marco@PC:/usr/share/zookeeper$ sudo kill 4545


marco@PC:/usr/local/kafka$ bin/zookeeper-server-start.sh   
config/zookeeper.properties
[2018-09-04 12:18:14,608] INFO Reading configuration from:  
config/zookeeper.properties  
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2018-09-04 12:18:14,609] INFO autopurge.snapRetainCount set to 3  
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 12:18:14,609] INFO autopurge.purgeInterval set to 0  
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 12:18:14,609] INFO Purge task is not scheduled. 
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 12:18:14,610] WARN Either no config or no quorum defined in  
config, running  in standalone mode    
(org.apache.zookeeper.server.quorum.QuorumPeerMain)
[2018-09-04 12:18:14,620] INFO Reading configuration from:  
config/zookeeper.properties   
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2018-09-04 12:18:14,620] INFO Starting server       
(org.apache.zookeeper.server.ZooKeeperServerMain)
[2018-09-04 12:18:14,624] INFO Server  
environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03 
built on 06/29/2018 00:39 GMT (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,624] INFO Server environment:host.name=PC.station  
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.version=1.8.0_181 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.vendor=Oracle 
Corporation (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.home=/usr/lib/jvm
/java-8-oracle/jre (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.class.path=/usr/local
/kafka/bin/../libs/activation-1.1.1.jar:/usr/local/kafka/bin/..
/libs/aopalliance-repackaged-2.5.0-b42.jar:/usr/local/kafka/bin/..
/libs/argparse4j-0.7.0.jar:/usr/local/kafka/bin/../libs/audience-annotations-
0.5.0.jar:/usr/local/kafka/bin/../libs/commons-lang3-3.5.jar:/usr/local/kafka
/bin/../libs/connect-api-2.0.0.jar:/usr/local/kafka/bin/../libs/connect-
basic-auth-extension-2.0.0.jar:/usr/local/kafka/bin/../libs/connect-
file-2.0.0.jar:/usr/local/kafka/bin/../libs/connect-json-2.0.0.jar:/usr/local
/kafka/bin/../libs/connect-runtime-2.0.0.jar:/usr/local/kafka/bin/..
/libs/connect-transforms-2.0.0.jar:/usr/local/kafka/bin/../libs/guava-
20.0.jar:/usr/local/kafka/bin/../libs/hk2-api-2.5.0-b42.jar:/usr/local/kafka
/bin/../libs/hk2-locator-2.5.0-b42.jar:/usr/local/kafka/bin/../libs/hk2-
utils-2.5.0-b42.jar:/usr/local/kafka/bin/../libs/jackson-annotations-
2.9.6.jar:/usr/local/kafka/bin/../libs/jackson-core-2.9.6.jar:/usr/local
/kafka/bin/../libs/jackson-databind-2.9.6.jar:/usr/local/kafka/bin/..
/libs/jackson-jaxrs-base-2.9.6.jar:/usr/local/kafka/bin/../libs/jackson-
jaxrs-json-provider-2.9.6.jar:/usr/local/kafka/bin/../libs/jackson-module-
jaxb-annotations-2.9.6.jar:/usr/local/kafka/bin/../libs/javassist-3.22.0-
CR2.jar:/usr/local/kafka/bin/../libs/javax.annotation-api-1.2.jar:/usr/local
/kafka/bin/../libs/javax.inject-1.jar:/usr/local/kafka/bin/.. 
/libs/javax.inject-2.5.0-b42.jar:/usr/local/kafka/bin/../libs/javax.servlet-
api-3.1.0.jar:/usr/local/kafka/bin/../libs/javax.ws.rs-api-2.1.jar:/usr/local
/kafka/bin/../libs/jaxb-api-2.3.0.jar:/usr/local/kafka/bin/../libs/jersey-
client-2.27.jar:/usr/local/kafka/bin/../libs/jersey-common-2.27.jar:
/usr/local/kafka/bin/../libs/jersey-container-servlet-2.27.jar:/usr/local
/kafka/bin/../libs/jersey-container-servlet-core-2.27.jar:/usr/local/kafka
/bin/../libs/jersey-hk2-2.27.jar:/usr/local/kafka/bin/../libs/jersey-media-
jaxb-2.27.jar:/usr/local/kafka/bin/../libs/jersey-server-2.27.jar:/usr/local
/kafka/bin/../libs/jetty-client-9.4.11.v20180605.jar:/usr/local/kafka/bin/..
/libs/jetty-continuation-9.4.11.v20180605.jar:/usr/local/kafka/bin/..
/libs/jetty-http-9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-
io-9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-security-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-server-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-servlet-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-servlets-
9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jetty-
util-9.4.11.v20180605.jar:/usr/local/kafka/bin/../libs/jopt-simple-5.0.4.jar:
/usr/local/kafka/bin/../libs/kafka_2.11-2.0.0.jar:/usr/local/kafka/bin/..
/libs/kafka_2.11-2.0.0-sources.jar:/usr/local/kafka/bin/../libs/kafka-
clients-2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-log4j-appender-
2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-streams-2.0.0.jar:/usr/local
/kafka/bin/../libs/kafka-streams-examples-2.0.0.jar:/usr/local/kafka/bin/..
/libs/kafka-streams-scala_2.11-2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-
streams-test-utils-2.0.0.jar:/usr/local/kafka/bin/../libs/kafka-tools-
2.0.0.jar:/usr/local/kafka/bin/../libs/log4j-1.2.17.jar:/usr/local/kafka
/bin/../libs/lz4-java-1.4.1.jar:/usr/local/kafka/bin/../libs/maven-artifact-
3.5.3.jar:/usr/local/kafka/bin/../libs/metrics-core-2.2.0.jar:/usr/local
/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/usr/local/kafka/bin/..
/libs/plexus-utils-3.1.0.jar:/usr/local/kafka/bin/../libs/reflections-
0.9.11.jar:/usr/local/kafka/bin/../libs/rocksdbjni-5.7.3.jar:/usr/local/kafka
/bin/../libs/scala-library-2.11.12.jar:/usr/local/kafka/bin/../libs/scala-
logging_2.11-3.9.0.jar:/usr/local/kafka/bin/../libs/scala-reflect-
2.11.12.jar:/usr/local/kafka/bin/../libs/slf4j-api-1.7.25.jar:/usr/local
/kafka/bin/../libs/slf4j-log4j12-1.7.25.jar:/usr/local/kafka/bin/..
/libs/snappy-java-1.1.7.1.jar:/usr/local/kafka/bin/../libs/validation-
api-1.1.0.Final.jar:/usr/local/kafka/bin/../libs/zkclient-0.10.jar:/usr/local
/kafka/bin/../libs/zookeeper-3.4.13.jar 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.library.path=/usr
/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib   
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.io.tmpdir=/tmp
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:java.compiler=<NA>  
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:os.name=Linux  
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:os.arch=amd64 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:os.version=4.15.0-33-
generic (org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:user.name=marco 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:user.home=/home/marco 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,625] INFO Server environment:user.dir=/usr/local/kafka 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,630] INFO tickTime set to 3000 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,630] INFO minSessionTimeout set to -1 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,630] INFO maxSessionTimeout set to -1 
(org.apache.zookeeper.server.ZooKeeperServer)
[2018-09-04 12:18:14,634] INFO Using  
org.apache.zookeeper.server.NIOServerCnxn Factory as server connection  
factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2018-09-04 12:18:14,636] INFO binding to port 0.0.0.0/0.0.0.0:2181   
(org.apache.zookeeper.server.NIOServerCnxnFactory)

The problem is that rebooting zookeeper seems active:

marco@PC:~$ netstat -ant | grep :2181
tcp6       0      0 :::2181                 :::*                      
LISTEN
marco@PC:~$ systemctl status zookeeper
● zookeeper.service - LSB: centralized coordination service
   Loaded: loaded (/etc/init.d/zookeeper; generated)
   Active: active (running) since Tue 2018-09-04 15:51:24 CEST; 2min 19s
   ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1919 ExecStart=/etc/init.d/zookeeper start (code=exited, 
status=0/SUCCESS)
    Tasks: 25 (limit: 4915)
   CGroup: /system.slice/zookeeper.service
           └─1997 /usr/bin/java -cp /etc/zookeeper/conf:/usr/share
/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share   
/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share
/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr

Sep 04 15:51:24 PC systemd[1]: Starting LSB: centralized coordination 
service...
Sep 04 15:51:24 PC systemd[1]: Started LSB: centralized coordination 
service.

But when trying to run kafka... :

marco@PC:~$ cd /usr/local/kafka
marco@PC:/usr/local/kafka$ bin/zookeeper-server-start.sh   
config/zookeeper.properties
[2018-09-04 15:54:29,981] INFO Reading configuration from:  
config/zookeeper.properties  
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2018-09-04 15:54:29,982] INFO autopurge.snapRetainCount set to 3  
(org.apache.zookeeper.server.DatadirCleanupManager)
[2018-09-04 15:54:29,983] INFO autopurge.purgeInterval set to 0 
(org.apache.zookeeper.server.DatadirCleanupManager)
...
[2018-09-04 15:54:30,012] INFO binding to port 0.0.0.0/0.0.0.0:2181  
(org.apache.zookeeper.server.NIOServerCnxnFactory)
[2018-09-04 15:54:30,013] ERROR Unexpected exception, exiting abnormally 
(org.apache.zookeeper.server.ZooKeeperServerMain)
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:433)
        at sun.nio.ch.Net.bind(Net.java:425)
        at 

sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
apache-kafka apache-zookeeper port
2 Answers
$ bin/zookeeper-server-start.sh  config/zookeeper.properties

This started okay if you look at the logs

[2018-09-04 12:18:14,636] INFO binding to port 0.0.0.0/0.0.0.0:2181
(org.apache.zookeeper.server.NIOServerCnxnFactory)

And this message will sit there until a Zookeeper client such as Kafka connects to Zookeeper.

When you say you're "trying to run kafka", you're running zookeeper-server-start again, not kafka-server-start, so Zookeeper is already bound to port 2181 and it won't start a second one.

user_23086830

Maybe I "solved" the problem:

I left opened the "idle" window ( First Window ), I opened a new window and started the kafka server : Step 2.b of ApachaKafka-QuickStart : /usr/local/ kafka/ bin/kafka-server-start.sh config/server.properties ( Second Window ). Then I opened another window to create a topic and send some messages: 

marco@PC:/usr/local/kafka$ bin/kafka-topics.sh --create --zookeeper localhost:2181   
--replication-factor 1 --partitions 1 --topic test
Created topic "test".
marco@PC:/usr/local/kafka$ bin/kafka-topics.sh --list --zookeeper localhost:2181
test
marco@PC:/usr/local/kafka$ bin/kafka-console-producer.sh --broker-list localhost:9092   
--topic test
>This is a message
>This is another message
>

( ThirdWindow ) . Finally, in another new window I started a consumer :

marco@PC:/usr/local/kafka$ bin/kafka-console-consumer.sh --bootstrap-server  
localhost:9092 --topic test --from-beginning
This is a message
This is another message

: ( FourthWindow )

What I actually do not understand now is why I should open all these four sessions.

Marco

user_23150940
Related