k8s elasticsearch pod running but not work with dynamic pvc

Question

I use https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner and everything is work.

But when I use helm to install elasticsearch, elasticsearch show running but not work.

Step I install elasticsearch

Create certificate , username/password for elasticsearch
manually create pvc/automatically create pvc by elasticsearch's value.yaml
elasticsearch show running but can't curl http://service's ip:9200

I also change the value of persistence: enabled: true to persistence: enabled: false and everything work , also can curl http://service's ip:9200 get the default response for elasticsearch

So I wonder if I misunderstand the way to use pvc, Here is my pvc.yaml and my status when I create pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: elasticsearch-master-elasticsearch-master-0
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 2Gi
  storageClassName: nfs-client

NAME                                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
elasticsearch-master-elasticsearch-master-0   Bound    pvc-55480402-c7d7-4a52-aa85-961f97ab7f82   2Gi        RWO            nfs-client     20m

My disk information for node that run es

10.10.1.134:/mnt/nfs/default-elasticsearch-master-elasticsearch-master-0-pvc-55480402-c7d7-4a52-aa85-961f97ab7f82   15G  1.8G   14G  12% /var/lib/kubelet/pods/b3696f5a-5d9b-4c00-943e-027c2dc7a86c/volumes/kubernetes.io~nfs/pvc-55480402-c7d7-4a52-aa85-961f97ab7f82

drwxrwxrwx 3 root root 19 Sep 19 17:25 pvc-55480402-c7d7-4a52-aa85-961f97ab7f82

My description of elasticsearch

Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  21m   default-scheduler  0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.
  Normal   Scheduled         21m   default-scheduler  Successfully assigned default/elasticsearch-master-0 to a136
  Normal   Pulled            21m   kubelet            Container image "docker.elastic.co/elasticsearch/elasticsearch:7.17.3" already present on machine
  Normal   Created           21m   kubelet            Created container configure-sysctl
  Normal   Started           21m   kubelet            Started container configure-sysctl
  Normal   Pulled            21m   kubelet            Container image "docker.elastic.co/elasticsearch/elasticsearch:7.17.3" already present on machine
  Normal   Created           21m   kubelet            Created container elasticsearch
  Normal   Started           21m   kubelet            Started container elasticsearch

My logs of es

{"type": "server", "timestamp": "2022-09-19T09:25:49,509Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "loaded module [x-pack-voting-only-node]" }
{"type": "server", "timestamp": "2022-09-19T09:25:49,509Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "loaded module [x-pack-watcher]" }
{"type": "server", "timestamp": "2022-09-19T09:25:49,510Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "no plugins loaded" }
{"type": "deprecation.elasticsearch", "timestamp": "2022-09-19T09:25:49,515Z", "level": "CRITICAL", "component": "o.e.d.c.s.Settings", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "[node.ml] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.", "key": "node.ml", "category": "settings" }
{"type": "deprecation.elasticsearch", "timestamp": "2022-09-19T09:25:49,621Z", "level": "CRITICAL", "component": "o.e.d.c.s.Settings", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "[node.data] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.", "key": "node.data", "category": "settings" }

curl when persist value to true

curl -u username:password http://10.101.45.49:9200
curl: (7) Failed connect to 10.101.45.49:9200; Connection refused

curl when persist value to false

curl -u username:password http://10.101.45.49:9200

{
  "name" : "elasticsearch-master-0",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "WRR3PXhLS7GGHl5AaUz2DA",
  "version" : {
    "number" : "7.17.3",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "5ad023604c8d7416c9eb6c0eadb62b14e766caff",
    "build_date" : "2022-04-19T08:11:19.070913226Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

My volumeClaimTemplate in elasticsearch value.yaml

volumeClaimTemplate:
  storageClassName: "nfs-client"
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 2Gi

It seems that es can't work when there is dynamic pvc, but i don't know how to solve the problem, thank you for help

k8s elasticsearch pod running but not work with dynamic pvc

0 Answers