Elasticsearch aggregation when a field does not exist takes a long time

I want to know the reason why, when a field does not exist, it takes a long time to aggregate. For example, sometimes a log doesn't have status code. When I want to know the distribution of all the status codes, it takes a long time. However, if I add the query condition that the status code must exist, it only takes a little time.