KAFKA - ERROR Disk error while locking directory

user_10215985 0
Viewed 8807

I got this error ERROR Disk error while locking directory while trying to start kafka-server-start.sh config/server.properties

[2020-05-21 23:44:11,323] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2020-05-21 23:44:11,323] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2020-05-21 23:44:11,324] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2020-05-21 23:44:11,340] ERROR Disk error while locking directory /opt/kafka2.13/kafka_2.13-2.5.0/data_log_tu_tao/kafka_data (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: /opt/kafka2.13/kafka_2.13-2.5.0/data_log_tu_tao/kafka_data/.lock
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
    at java.nio.channels.FileChannel.open(FileChannel.java:287)
    at java.nio.channels.FileChannel.open(FileChannel.java:335)
    at kafka.utils.FileLock.<init>(FileLock.scala:31)
    at kafka.log.LogManager.$anonfun$lockLogDirs$1(LogManager.scala:235)
    at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:118)
    at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:105)
    at scala.collection.mutable.ArraySeq.flatMap(ArraySeq.scala:38)
    at kafka.log.LogManager.lockLogDirs(LogManager.scala:233)
    at kafka.log.LogManager.<init>(LogManager.scala:104)
    at kafka.log.LogManager$.apply(LogManager.scala:1084)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:253)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
    at kafka.Kafka$.main(Kafka.scala:82)
    at kafka.Kafka.main(Kafka.scala)
[2020-05-21 23:44:11,344] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.nio.file.AccessDeniedException: /opt/kafka2.13/kafka_2.13-2.5.0/data_log_tu_tao/kafka_data/recovery-point-offset-checkpoint
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
    at java.nio.file.Files.newByteChannel(Files.java:361)
    at java.nio.file.Files.createFile(Files.java:632)
    at kafka.server.checkpoints.CheckpointFile.<init>(CheckpointFile.scala:45)
    at kafka.server.checkpoints.OffsetCheckpointFile.<init>(OffsetCheckpointFile.scala:57)
    at kafka.log.LogManager.$anonfun$recoveryPointCheckpoints$1(LogManager.scala:106)
    at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
    at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
    at scala.collection.mutable.ArraySeq.map(ArraySeq.scala:38)
    at kafka.log.LogManager.<init>(LogManager.scala:105)
    at kafka.log.LogManager$.apply(LogManager.scala:1084)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:253)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
    at kafka.Kafka$.main(Kafka.scala:82)
    at kafka.Kafka.main(Kafka.scala)
apache-kafka
3 Answers
user_33466540

For people using Kafka on windows and having a related error to

java.nio.file.AccessDeniedException:

This is a common error when log retention happens. Kafka doesn’t have good support for windows filesystem. You can use WSL2 or Docker to work around these limitations
Try to use kafka_2.12-2.8.1.tgz. This resolved the issue for me.

user_156626400

As the error states, the user that starts the Kafka Server process does not have access to your log.dirs: 

[2020-05-21 23:44:11,344] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.nio.file.AccessDeniedException: /opt/kafka2.13/kafka_2.13-2.5.0/data_log_tu_tao/kafka_data/recovery-point-offset-checkpoint

You can either:

  • Change log.dirs (Make sure NOT to use /tmp/)
  • Or grant read/write access for /opt/kafka2.13/kafka_2.13-2.5.0/data_log_tu_tao/kafka_data/

If none of the above options works for you, then probably it might be worth checking if the directory actually exists. If not, simply create it by running

mkdir -p /opt/kafka2.13/kafka_2.13-2.5.0/data_log_tu_tao/kafka_data

As a side note, I wouldn't say that /opt/ is the best place to store data.

user_71317570
Related