Using unordered_map Causes Time Limit Exceeded But map Gets Accepted

I was doing this problem for practice. Here is my initial solution:

#include <bits/stdc++.h>

using namespace std;
using ll = long long;

int main() {
  ios::sync_with_stdio(0);
  cin.tie(0);

  int n, x;
  cin >> n >> x;

  unordered_map<ll, int> cnt;
  ll sum = 0;
  ll res = 0;
  cnt[0] = 1;

  while(n --) {
    int a;
    cin >> a;
    sum += a;
    res += cnt[sum - x];
    cnt[sum] ++;
  }

  cout << res;
}

The solution is pretty simple, using a prefix sum and for each element check how many times the complement number occurred before. Here, I use unordered_map in order to count the frequency of each element, as a regular bitmap would be far too large. However, when submitting this I got time limit exceeded on a few test cases. I was stumped for a long time, so I had to look at the solution. The solution was very similar to mine, but there was a difference in that they used map while I use unordered_map. So I changed which map I used, and when submitting again it got accepted. I am curious as to why using a map is faster than unordered_map in this situation, as I thought map was slower due to it's nature of sorting all elements coming in.