Django Rest Framework: How to use EXCEPTION_HANDLER with DEBUG = True?

I'm using a custom exception handler for my REST API, which responds with JSON data that my front end knows how to handle. The problem is that with the setting DEBUG = True, Django doesn't use my custom exception handler. Instead, it responds with the standard HTML debug page, like this:

Here's my settings.py:

DEBUG = True

REST_FRAMEWORK = {
   # This doesn't seem to do anything when DEBUG == True
   'EXCEPTION_HANDLER': 'stripe_app.utils.custom_exception_handler'
}
...

I know that I could just set DEBUG = False, but if I did that I'd also have to change many other things that use the DEBUG variable.

So, How can I use my custom exception handler with the DEBUG = True setting?

I've already looked for answers in the DRF docs: https://www.django-rest-framework.org/api-guide/settings/#exception_handler

and the Django docs: https://docs.djangoproject.com/en/3.1/ref/settings/#debug

but I can't find the answer I'm looking for.

EDIT

I just realized that I forgot to set the following in my root urls.py, per the DRF docs:

handler400 = 'stripe_app.views.bad_request'
handler500 = 'stripe_app.views.server_error'

However, DEBUG = True still causes problems. For example, even if I created a view like this:

def always_raise_exception(request):
    raise Exception

If DEBUG == True, the above view responds with the standard Django HTML debug page.

If DEBUG == False, it responds (correctly) with a JSON response (another edit: I think this is just because the server server raised a different exception than with DEBUG = True).