nginx/gunicorn/flask returning 499/502 errors when more than 5 event streams are open

I am using AWS Elastic Beanstalk to serve a backend for a mobile app on iOS/Android. I have been encountering an error when the server has to deal with 6 or more event streams (used for SSE purposes). When less than 6 event streams are open, then the others work completely as expected. These event streams are not from the same device/browser, and only a total of 6 is needed to start causing issues. Once there are 6 endpoints listening for SSEs, every single other route returns a 499 error. This essentially bricks the server entirely, and even when the event streams are closed, the server still will always show 499 errors. Currently, the timeout is set to 24h, but if this was removed, these routes return a 502 error after 60s. I have attached my nginx configuration, the route that is used to send the event stream, and a testing listener client (which should be equivalent to using EventSource in JavaScript). I am heavily leaning towards just using WebSockets, because SSE seems like it will continue to cause me nightmares in the future.

#Elastic Beanstalk Nginx Configuration File

user                    nginx;
error_log               /var/log/nginx/error.log warn;
pid                     /var/run/nginx.pid;

worker_processes        auto;
worker_rlimit_nofile    32804;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    include       conf.d/*.conf;

    # This specific aspect is for SSE (allows open connections to stay open)
    proxy_read_timeout 24h;
    proxy_connect_timeout 120;
    proxy_send_timeout 120;
    proxy_buffering off;
    proxy_cache off;

    proxy_set_header Connection '';
    proxy_http_version 1.1;
    chunked_transfer_encoding off;

    map $http_upgrade $connection_upgrade {
        default     "upgrade";
    }

    server {
        listen        80 default_server;
        access_log    /var/log/nginx/access.log main;

        client_header_timeout 60;
        client_body_timeout   60;
        keepalive_timeout     60;
        gzip                  off;
        gzip_comp_level       4;
        gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        # Include the Elastic Beanstalk generated locations
        include conf.d/elasticbeanstalk/*.conf;
    }
}

@notification.route('/listen', methods=['GET'])
@auth_required()
def _listen():

    # Get the user_id out here, since request is inaccessible in stream()
    user_id = get_auth_user_id()

    def stream():
        # Listen for messages
        messages = sse.sse_listen(user_id)
        while True:
            try:
                msg = messages.get()  # blocks until a new message arrives
                yield msg
            except:
                return GatewayTimeout("Queue emptied of messages.")

    # Return an SSE stream
    headers = {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-transform',
        'Connection': 'keep-alive',
        'Accept-Encoding': '*',
        'X-Accel-Buffering': 'no'
    }

    return Response(stream(), headers=headers)

import sseclient

headers={
    'Authorization': 'Bearer <long key>'
}
url = 'https://testapi.hotspotparty.co/v1/user/notification/listen'
messages = sseclient.SSEClient(url, headers=headers)

for message in messages:
    print(message)