Elasticsearch must clause faster than filter

We use elasticsearch 7.2 and we've been observing something weird lately

We tried executing the following two queries

{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "customer(keyword_field)": "big_customer"
          }
        }
      ]
    }
  }
}

{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "customer(keyword_field)": "big_customer"
          }
        }
      ]
    }
  }
}

This matches around ~1million documents. The 1st one was faster than the 2nd (10 times faster!). I expected 1 to be slower because of scoring

Also, when i added sorting, both of them got slower (2nd remained the same, 1st became as slow as 2nd)