Unassigned es-monitoring shards on ealstic search and single-node mode

user_288568 0
Viewed 323

We using elastic search on a single node and unassigned monitoring shards are piling up.

Where do those shards come from and how can they be avoided?

# curl http://localhost:9200/_cat/shards?pretty|grep UNASSIGNED
--:.monitoring-es-6-2021.02.08                                    0 r UNASSIGNED                           
--:.monitoring-es-6-2021.01.25                                    0 r UNASSIGNED                           
--.monitoring-es-6-2021.01.29                                    0 r UNASSIGNED                           
 --.monitoring-es-6-2021.02.17                                    0 r UNASSIGNED                           
:-.monitoring-es-6-2021.01.20                                    0 r UNASSIGNED                           
-:--.monitoring-es-6-2021.01.31                                    0 r UNASSIGNED                           
 -.monitoring-es-6-2021.02.01                                    0 r UNASSIGNED                           
-:-.monitoring-es-6-2021.02.09                                    0 r UNASSIGNED                           
-:.monitoring-es-6-2021.02.07                                    0 r UNASSIGNED                           
--.monitoring-es-6-2021.02.12                                    0 r UNASSIGNED                           
 13.monitoring-es-6-2021.02.15                                    0 r UNASSIGNED                           
18k.monitoring-es-6-2021.02.16                                    0 r UNASSIGNED                           

.monitoring-es-6-2021.01.19                                    0 r UNASSIGNED                           
.monitoring-es-6-2021.01.22                                    0 r UNASSIGNED                           
.monitoring-es-6-2021.02.18                                    0 r UNASSIGNED                           
.monitoring-es-6-2021.02.11                                    0 r UNASSIGNED                           
.monitoring-es-6-2021.01.18                                    0 r UNASSIGNED                           
.monitoring-es-6-2021.01.24                                    0 r UNASSIGNED                           

 ....

We start elasticsearch using a docker image, form a Dockerfile like this:

ARG version=5.6.16
FROM docker.elastic.co/elasticsearch/elasticsearch:${version}
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-phonetic \
    && /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-icu

With this env:

environment:
  - xpack.security.enabled=false
  - discovery.type=single-node
  - TAKE_FILE_OWNERSHIP=true

We found https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/ but it seems to solve this only temporarily and would be needed to be executed regularily. We try to avoid a hacky cron job solution and want to solve the root cause :-)

elasticsearch-5
2 Answers
user_136377600

Check https://discuss.elastic.co/t/what-are-monitoring-indices/144855

Where do those shards come from?

"Those are system indices created by Elasticsearch and Kibana, they're usually small and gets deleted after seven days so i wouldn't worry."

How can they be avoided?

"With 6.3 x-pack ships with any distribution by default and monitoring is enabled by default. If you really want to disable it, you can, see https://www.elastic.co/guide/en/elasticsearch/reference/current/monitoring-settings.html"

By index pattern, is about the way you create your indices.

For example, this process creates one index per day.

Other patterns:

  • Create an index by customer
  • Create an index by date
  • Create an index per year etc.

There will be those who only have one index and do not follow a pattern to create indices. The patterns for creating indices do not exist anywhere in ES. It is a pattern that you, as an ES user, follow to create your indices according to your needs.

user_82379700
Related