SignalR connection causes gateway timeout

I'm at a bit of a loss on this one, been debugging it for days and I'm baffled.

I have an ASP.NET 6 API with SignalR.

builder.Services.AddSignalR()
       .AddHubOptions<TestHub>(x =>
       {
           x.EnableDetailedErrors = true;
       })
       .AddStackExchangeRedis(dbCon.Redis, options =>
       {
           ...
       })
       .AddJsonProtocol(options =>
       {
           ...
       });
...
app.MapStatusEndpoints();
app.MapControllers();
app.MapHub<TestHub>("/api/h");

Locally this all works perfectly without issue.

When I throw it up on the server with nginx (AWS with t4g.small, so arm64), for some reason after a web socket connection is established, any api call from the same page causes nginx to return gateway timeouts, and causes SignalR to disconnect for roughly 1 minutes, the connection reconnects, api call fails, gateway timeouts again. Even if the api call is

I've looked through the logs on nginx and all I see is:

2022/09/24 16:18:30 [error] 41453#41453: *17989 connect() failed (111: Connection refused) while connecting to upstream, client: 10.30.4.89, server: test.SNIP.com, request: "GET /api/h?access_token=SNIP HTTP/1.1", upstream: "http:// 127.0.0.1:5010/api/h?access_token=SNIP", host: "test.SNIP.com"

2022/09/24 16:25:31 [error] 41453#41453: *18087 connect() failed (111: Connection refused) while connecting to upstream, client: 10.30.4.89, server: test.SNIP.com, request: "POST /api/start HTTP/1.1", upstream: "http:// 127.0.0.1:5010/api/start", host: "test.SNIP.com", referrer: "https: //test.SNIP.com/arena"

I think* it's Kestrel/SignalR rejecting the request, but I don't understand why, I can't find any logs to indicate why...

I more or less trimmed my nginx site back to almost bare minimum...

server {
    listen        80;
    server_name   test.*snip*.com;
    server_tokens off;
    
    location / {
        root /var/www/test.*snip*.com/deployments/0.0.0/vue;
        index index.html;
        try_files $uri /index.html;
    }

    location /api/ {
        proxy_pass         http://127.0.0.1:5010;
        proxy_http_version 1.1;
        proxy_set_header   Upgrade $http_upgrade;
        proxy_set_header   Connection 'keep-alive, upgrade';
        proxy_set_header   Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_set_header   Host $host;
        proxy_cache off;
        proxy_buffering off;
    }
}

A solution would be nice, but looking for idea's on how to figure out what the problem is. No exceptions are raised from the app.

Update: I found the journalctl logs and it appears that System.Text.Json causes a StackOverflow when on a t4g.small instance.

If I keep STJ and use an amd instance (t3a.small) it works.

If I change to newtonsoft.json and use a t4g.small it works.

But STJ + t4g.small it fails when using Dictionary<string, object>

The JSON looks like:

{
    "action": "Battle",
    "data": {
        "Level": 10
    }
}

And the class

public sealed class Action
{
    public int Id { get; set; }
    public ActionTypes ActionType { get; set; } // enum
    public int UserId { get; set; }
    public Dictionary<string, string> Data { get; set; } = new();
}

So it's nothing out of the ordinary... Unsure why STJ throws up on this.