I want to write a script to health check our elasticsearch cluster (deploy on kubernetes)
- I go inside pod which run elasticsearch master container and run below commands:
[elasticsearch@elasticsearch-master-0 ~]$ curl localhost:9200/frontend-dev-2021.12.03/_count
{"count":76,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}
[elasticsearch@elasticsearch-master-0 ~]$ curl localhost:9200/_cluster/health?pretty
{
"cluster_name" : "elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 617,
"active_shards" : 1234,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
As you can see, both index count and health check command are success. But when I run these command from outside (I give elasticsearch cluster an public endpoint)
root@ip-192-168-1-1:~# curl --user username:password esdev.example.com/frontend-dev-2021.12.03/_count
{"count":76,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}
root@ip-192-168-1-1:~# curl --user username:password esdev.example.com/_cluster/health
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
Only the index count command is success, the health check command always produce 403 Forbidden error.
I have searched and read through the official docs from elasticsearch but event the offcial docs only run command internal elasticsearch cluster or using kibana (http service kubernetes - internal k8s cluster).
How can I health check elasticsearch from outside? Or we can not do this because some mechanism of elasticsearch cluster?
Notes: I create a basic auth nginx (username:password) stand before the elasticsearch and this nginx has an ingressroute from traefik-v2
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
annotations:
meta.helm.sh/release-name: basic-auth-nginx-dev
meta.helm.sh/release-namespace: dev
creationTimestamp: "2021-01-23T08:12:55Z"
generation: 2
labels:
app: basic-auth-nginx-dev
app.kubernetes.io/managed-by: Helm
managedFields:
- apiVersion: traefik.containo.us/v1alpha1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:meta.helm.sh/release-name: {}
f:meta.helm.sh/release-namespace: {}
f:labels:
.: {}
f:app: {}
f:app.kubernetes.io/managed-by: {}
f:spec:
.: {}
f:entryPoints: {}
f:routes: {}
manager: Go-http-client
operation: Update
time: "2021-01-23T08:12:55Z"
name: basic-auth-nginx-dev-web
namespace: dev
resourceVersion: "103562796"
selfLink: /apis/traefik.containo.us/v1alpha1/namespaces/dev/ingressroutes/basic-auth-nginx-dev-web
uid: 5832b501-b2d7-4600-93b6-b3c72c420115
spec:
entryPoints:
- web
routes:
- kind: Rule
match: Host(`esdev.example.com`) && PathPrefix(`/`)
priority: 1
services:
- kind: Service
name: basic-auth-nginx-dev
port: 80