Unexpected Disk Capacity issue using Elasticsearch cluster on EKS with a 8Exabytes EFS disk

I have configured an elasticsearch cluster in my kubernetes cluster (EKS), the elasticsearch cluster has 3 nodes and I have set up a 8E disk for the nodes to store the data. (thinking that I wont have any space issues for a while...)

[root@es-cluster-0 elasticsearch]# curl -s -XGET http://localhost:9200/_cat/allocation?v
shards disk.indices disk.used disk.avail disk.total disk.percent host         ip           node
    36       66.7gb   966.1gb   8191.9pb   8191.9pb            0 10.65.32.184 10.65.32.184 es-cluster-0
    33       82.6gb   966.1gb   8191.9pb   8191.9pb            0 10.65.32.202 10.65.32.202 es-cluster-2
    37         76gb   966.1gb   8191.9pb   8191.9pb            0 10.65.32.178 10.65.32.178 es-cluster-1
    14                                                                                     UNASSIGNED

The cluster current health is:

[root@es-cluster-0 elasticsearch]# curl -s -XGET http://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "k8s-logs",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 56,
  "active_shards" : 106,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 14,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 88.33333333333333
}

I can see that I have 14 "unassigned_shards" which matches perfectly with the last line of the /_cat/allocation above

When I'm start figuring out what is happening I found this:

[root@es-cluster-0 elasticsearch]# curl -s -XGET http://localhost:9200/_cluster/allocation/explain?pretty
{
  "index" : "logstash-2022.01.22",
  "shard" : 0,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "ALLOCATION_FAILED",
    "at" : "2022-01-22T00:00:11.254Z",
    "failed_allocation_attempts" : 5,
    "details" : "failed shard on node [bf_GjmcUQGuCTk-_voh4Xw]: failed recovery, failure RecoveryFailedException[[logstash-2022.01.22][0]: Recovery failed from {es-cluster-0}{hYJ4ifx7R7yWJq6VFP3Drw}{jjAAtdcmQXeVpJXxj4DYcA}{10.65.32.184}{10.65.32.184:9300}{dilmrt}{ml.machine_memory=15878057984, ml.max_open_jobs=20, xpack.installed=true, transform.node=true} into {es-cluster-1}{bf_GjmcUQGuCTk-_voh4Xw}{QNp4DD51TQa716D4TjMFPg}{10.65.32.178}{10.65.32.178:9300}{dilmrt}{ml.machine_memory=15878057984, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]; nested: RemoteTransportException[[es-cluster-0][10.65.32.184:9300][internal:index/shard/recovery/start_recovery]]; nested: RemoteTransportException[[es-cluster-1][10.65.32.178:9300][internal:index/shard/recovery/clean_files]]; nested: UncategorizedExecutionException[Failed execution]; nested: NotSerializableExceptionWrapper[execution_exception: java.io.IOException: Disk quota exceeded]; nested: IOException[Disk quota exceeded]; ",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "7WHft5LVTYCEWvwKM64A-w",
      "node_name" : "es-cluster-2",
      "transport_address" : "10.65.32.202:9300",
      "node_attributes" : {
        "ml.machine_memory" : "15878057984",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "transform.node" : "true"
      },         
--- TRUNCATED ---

I don't know why it's saying Disk quota exceeded if the elasticsearch cluster is reporting correctly the capacity that it has available /_cat/allocation is there any additional configuration that I need to setup in order to tell elasticsearch that we have enough space to work with ?