NGINX Reverse Proxy Fails with 404 despite being able to curl endpoint

I'm trying to reverse proxy to another endpoint on /alertmanager but it fails to connect. Weirdly enough I'm able to connect the endpoint directly from inside the pod running nginx.

A quick overview of my application architecture is this:

nginx ingress on cluster-> nginx load balancer -> <many services on different endpoints>

This is a minimized nginx configuration that replicates the issue:

worker_processes  5;  ## Default: 1
error_log  /dev/stderr;
pid        /tmp/nginx.pid;
worker_rlimit_nofile 8192;

events {}

http {
    client_body_temp_path /tmp/client_temp;
    proxy_temp_path       /tmp/proxy_temp_path;
    fastcgi_temp_path     /tmp/fastcgi_temp;
    uwsgi_temp_path       /tmp/uwsgi_temp;
    scgi_temp_path        /tmp/scgi_temp;

    default_type application/octet-stream;
    log_format   main '$remote_addr - $remote_user [$time_local]  $status '
        '"$request" $body_bytes_sent "$http_referer" '
        '"$http_user_agent" "$http_x_forwarded_for"';
    access_log   /dev/stderr  main;

    sendfile     on;
    tcp_nopush   on;
    resolver kube-dns.kube-system.svc.cluster.local;

    server {
        listen       8080;
        proxy_set_header X-Scope-OrgID 0;

        location = / {
            return 200 'OK';
            auth_basic off;
        }

        location /alertmanager {
            proxy_pass   http://mimir-distributed-alertmanager.mimir.svc.cluster.local:8080$request_uri;
        }
    }
}

I'm able to curl to the mimir endpoint in /alertmanager but I can't reach /alertmanager without getting a 404 error but I can get to / and if I put the proxy_pass inside of / it does work.

Example of what I'm seeing:

/ $ curl localhost:8080/
OK
/ $ curl localhost:8080/alertmanager
the Alertmanager is not configured

Curling http://mimir-distributed-alertmanager.mimir.svc.cluster.local does infact return the html of the page I'm expecting