After upgrading react-scripts, I get WebSocket connection to 'wss://example.com:3000/ws' failed

Here's my tech stack:

• Ubuntu
• Nginx
• Docker
• React
• mkcert

I use mkcert to create valid local certificates. Then instead of localhost:3000, I use hosts file to create a DNS record for my project. Thus I develop using https://project.local domain.

And I use nginx as a reverse proxy and here's my nginx config file:

server {
    listen 80;
    server_name project.local;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl;
    server_name project.local;

    ssl_certificate /Temp/Project/AdminPanel/Certificate.pem;
    ssl_certificate_key /Temp/Project/AdminPanel/Key.pem;

    error_page 502 /502.html;

    location /502 {
        root /Temp/Project/AdminPanel/Nginx;
        internal;
    }

    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
    }
}

It was working great. But when I upgraded my react-scripts from v4 to v5, it now complains that:

WebSocketClient.js:13 WebSocket connection to 'wss://project.local:3000/ws' failed:    
WebSocketClient @   WebSocketClient.js:13   
initSocket  @   socket.js:17   
(anonymous) @   socket.js:38   

What should I change to fix this?