I am aware of ES behavior that if any node is down then Elasticsearch just promotes that replica shard to primary shard(instant) and then see if there is another node in the cluster where it can copy the replica shard. But I am assuming this process must be taking some seconds. So I have 2 below questions:
Requests coming to ES cluster meanwhile (when ES was promoting another replica shard to primary shard), what happens to them?
- Does it fail?
- Does it continue retrying? If yes how many times? is it configured somewhere in ES code? is it based on a number of times? or is there any request timeout time is configured in ES? And what happens if still, ES didn't come back in this specific time?
Please Assume:
I have 2 nodes Node1 and Node2.
Node1: 1 primary shard
Node2: 1 replica shard
And Node1 is going down.