Cant access sub paths on Nginx + React router behind a proxy

I have an nginx server with a local web app running locally in port 4000. I was able to create the Nginx rules to have it loaded trough the proxy, but now I can only access the website trough the main url, like "https://app.domain.com" (this works well), if I try entering trough any link like "https://app.domain.com/page" I get a 404.

This is my current nginx config;

server {
  # gzip
  gzip on;
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_types text/plain text/css text/xml application/json application/javascript application/xml+rss application/atom+xml image/svg+xml;

  # SSL configuration
  listen 443;
  listen [::]:443;
  expires $expires;
  ssl on;
  ssl_certificate /etc/ssl/app.crt;
  ssl_certificate_key /etc/ssl/app.key;

  index index.html;

  server_name app.domain.com;
  proxy_intercept_errors on;
  autoindex off;

  root /var/www/domain-app;

  # I am using this rule to allow files like icons or css to be accessed directly
  location ~ ^.+\..+$ {
    proxy_set_header Host $host;
    proxy_pass http://localhost:4000;
    proxy_redirect off;

    if (!-e $request_filename){
      rewrite https://$server_name break;
    }
    
    try_files $uri /index.html;
  }
  
  location / {
    proxy_set_header Host $host;
    proxy_pass http://localhost:4000;
    proxy_redirect off;

    try_files $uri /index.html;
  }
}

I tried something like this on the location and it worked for root domain and for pages, but accessing the site from a sub-page like "https://app.domain.com/page/sub" failed; It loaded the page but it tried to find the assets under "https://app.domain.com/page/static/..."

location / {
  proxy_set_header Host $host;
  proxy_pass http://localhost:3000;
  proxy_redirect off;

  set $fallback_file /index.html;
  if ($http_accept !~ text/html) {
      set $fallback_file /null;
  }
  try_files $uri $fallback_file;
}