switch off html wrapping of errors in nginx

I use nginx on Ubuntu to forward requests to a Spring Boot-driven API server. Sometimes the API returns a body with 4xx and 5xx errors, that the client application consumes. E.g. in the API code I might return:

{
  "message": "blah"
}

However nginx seems to wrap errors up into some HTML and embeds the response within them. Is there any way to disable this behavior? And is there a way to do it en masse (i.e. for all 4xx and 5xx errors, without explicitly writing them all out with error_page for example)

Similar question and accepted answer here, however that seems to be a bit of hack. I'm sure there's a better way of doing this...

EDIT: Config looks like this:

server {
    listen 80;

    server_name my.domain.com;

    location / {
            proxy_pass http://127.0.0.1:9001;
    }
}