Troubles with docker compose + image elasticsearch:6.8.0

user_12541841 0
Viewed 3777

I was trying to upgrade the version from my elastic image from 5.6 to 6.8.0, but when I run ddev start the ES container was not getting up.

DDEV logs from the elasticsearch service

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=1
[2020-02-28T21:32:29,269][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.8.0.jar:6.8.0]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.8.0.jar:6.8.0]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.8.0.jar:6.8.0]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.8.0.jar:6.8.0]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.8.0.jar:6.8.0]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.8.0.jar:6.8.0]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.8.0.jar:6.8.0]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?

Additionally, I cannot use the mem_limit parameter because I'm using the latest version 3.0.

docker-compose.elasticsearch.yaml:

version: '3.6'
services:
  elasticsearch:
    container_name: ddev-${DDEV_SITENAME}-elasticsearch
    hostname: ${DDEV_SITENAME}-elasticsearch
    image: elasticsearch:6.8.0
    ports:
      - "9200"
      - "9300"
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - VIRTUAL_HOST=$DDEV_HOSTNAME
      - HTTP_EXPOSE=9200
    ulimits:
      memlock:
        soft: -1
        hard: -1
    labels:
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
      - ".:/mnt/ddev_config"
  web:
    links:
      - elasticsearch:elasticsearch

volumes:
  elasticsearch:
    name: "${DDEV_SITENAME}-elasticsearch"

What am I doing wrong or what else is missing?

Update

I've tried the following solution according to an solution posted here

 version: '3.6'
services:
  elasticsearch:
    container_name: ddev-${DDEV_SITENAME}-elasticsearch
    hostname: ${DDEV_SITENAME}-elasticsearch
    image: elasticsearch:6.8.0
    ports:
      - "9200"
      - "9300"
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xmx1024m -Xms1024m"
      - VIRTUAL_HOST=$DDEV_HOSTNAME
      - HTTP_EXPOSE=9200
      - node.max_local_storage_nodes=3
    ulimits:
      memlock:
        soft: -1
        hard: -1
    labels:
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
      - ".:/mnt/ddev_config"
  web:
    links:
      - elasticsearch:elasticsearch

volumes:
  elasticsearch:
    name: "${DDEV_SITENAME}-elasticsearch"

but now, throws me the following error:

enter image description here

ddev docker-compose elasticsearch docker
2 Answers

I just had the same issue while installing ES 7.6 on my ubuntu AWS instance, which already had another ES version installed. And if you look closer at the end of your error message it mentioned:

Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?

Please refer to more details about this setting in the official ES doc.

To resolve the issue, you need to set node.max_local_storage_nodes to more than 1 in your elasticsearch.yml file.

More information about the error is available in the Elasticsearch source code in org.elasticsearch.env.NodeEnvironment#NodeEnvironment method.

if (nodeLock == null) {
    final String message = String.format(
        Locale.ROOT,
        "failed to obtain node locks, tried [%s] with lock id%s;" +
            " maybe these locations are not writable or multiple nodes were started without increasing [%s] (was [%d])?",
        Arrays.toString(environment.dataFiles()),
        maxLocalStorageNodes == 1 ? " [0]" : "s [0--" + (maxLocalStorageNodes - 1) + "]",
        MAX_LOCAL_STORAGE_NODES_SETTING.getKey(),
        maxLocalStorageNodes);
    throw new IllegalStateException(message, lastException);
}
user_40394310

I had same issue, the solution that worked for me was to remove the volume that host the elasticsearch data completely. It will delete the node.lock, and the entire index, but you can recreate it again.

user_131231710
Related