I have 11 elasticsearch nodes 3 master node 6 data node and 2 coordinate node.We are running latest version of elasticsearch 7.13.2 we have installed metricbeat and configured in all elasticsearch node we are monitoring our ELK stack and we have observed that .monitoring-es-* indices has 200gb ,100,150gb and .monitoring-logstash-* has less amount of data size same with the kibana
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .monitoring-es-7-mb-2021.07.15 NPdkPbofRde5YWd50oCzAA 1 1 95287036 0 141.2gb 70.6gb
green open .monitoring-es-7-mb-2021.07.16 F2oy_3WVRY6tSdhaMp7ZEg 1 1 16711910 0 25.1gb 12.4gb
green open .monitoring-es-7-mb-2021.07.11 d1JChmtgTGmnFoORnIcA1Q 1 1 93133543 0 135.9gb 67.9gb
green open .monitoring-es-7-mb-2021.07.12 MYu5ozjiQGGjGFBI5fjcvQ 1 1 94136537 0 137.9gb 68.9gb
green open .monitoring-es-7-mb-2021.07.13 7eLRyUWgTS-dSFE3ad669A 1 1 95323641 0 139.9gb 69.9gb
green open .monitoring-es-7-mb-2021.07.14 w2RB_A1TS1SeUBebLUURkA 1 1 95287470 0 140.7gb 70.3gb
green open .monitoring-es-7-mb-2021.07.10 llAWKQJwQ_-2FZg4Dbc3iA 1 1 92770558 0 135gb 67.5gb
we have enable elasticsearch-xpack module in metricbeat
elasticsearch-xpack.yml
- module: elasticsearch
xpack.enabled: true
period: 10s
metricsets:
- cluster_stats
- index
- index_recovery
- index_summary
- node
- node_stats
- pending_tasks
- shard
hosts:
- "https://xx.xx.xx.xx:9200" #em1
- "https://xx.xx.xx.xx:9200" #em2
- "https://xx.xx.xx.xx:9200" #em3
- "https://xx.xx.xx.xx:9200" #ec1
- "https://xx.xx.xx.xx:9200" #ec2
- "https://xx.xx.xx.xx:9200" #ed1
- "https://xx.xx.xx.xx:9200" #ed2
- "https://xx.xx.xx.xx:9200" #ed3
- "https://xx.xx.xx.xx:9200" #ed4
- "https://xx.xx.xx.xx:9200" #ed5
- "https://xx.xx.xx.xx:9200" #ed6
scope: cluster
ssl.certificate_authorities: ["/etc/elasticsearch/certs/ca/ca.crt"]
username: "xxxx"
password: "********"
How can i reduce its size or which metricset should i monitor Is this normal behaviour?