Nginx ignoring /api location directive with react-router 4, except in firefox private browsing

I have a React app that is served via nginx, and a nodejs api server behind nginx reverse proxy. The nginx configuration looks like this:

location / {
    try_files $uri /index.html;
    add_header   Cache-Control public;
}

location /api/ {
    proxy_pass http://localhost:8000;
    proxy_http_version 1.1;
}

In firefox private browsing, things work as expected: when I page refresh/redirect to domain.com/api, the request gets proxied to the node server.

However, in non-private firefox and chrome (incognito + not), any page refresh/redirect to domain.com/api will load the react app and treat the /api as a react-router route. The strange thing is, if I clear cookies/history and direct my browser to domain.com/api, I will correctly be proxied to the node server. The issue only occurs after I have loaded the react app once before.

This is driving me crazy, any ideas? I was thinking about downgrading react-router to version 3, but that would require some refactoring and I don't know if that would solve things.