Why can I exceed the recursion limit?

Apparently I'm allowed to violate the recursion limit after all. It's 1000, but I can do 1020 no problem:

>>> import sys
>>> sys.getrecursionlimit()
1000
>>> def f(depth):
        if depth:
            f(depth - 1)

>>> f(1020)
>>>

Seems 1025 is the actual limit:

>>> import sys
>>> sys.getrecursionlimit()
1000
>>> depth = 0
>>> def f():
        global depth
        depth += 1
        f()

>>> f()
Traceback (most recent call last):
  File "<pyshell#8>", line 1, in <module>
    f()
  File "<pyshell#7>", line 4, in f
    f()
  File "<pyshell#7>", line 4, in f
    f()
  File "<pyshell#7>", line 4, in f
    f()
  [Previous line repeated 1022 more times]
RecursionError: maximum recursion depth exceeded
>>> depth
1025

Why am I allowed to exceed the limit?

I'm using CPython 3.8.5 32 bit on Windows 10 64 bit.

Update: I thought maybe I have an off-by-1 misinterpretation and it's actually 1024 = 2¹⁰. So I guessed maybe setting the limit to 2000 would allow 2048. But no, it's always the same constant number of extra recursions allowed:

>>> for limit in range(1000, 10001, 1000):
        sys.setrecursionlimit(limit)
        depth = 0
        try:
            f()
        except:
            print(limit, '=>', depth)

1000 => 1025
2000 => 2025
3000 => 3025
4000 => 4025
5000 => 5025
6000 => 6025

================================ RESTART: Shell ================================
>>>

Note that trying limit 7000 crashed so badly that the shell got restarted.