static/js/main.89d5e2e7.chunk.js net::ERR_ABORTED 404 (Not Found) unless I delete docker image before re-building

I am trying to get a production build for a React app using Docker. When I make a change to React, and try to re-build with docker-compose up --build, I get static/js/main.89d5e2e7.chunk.js net::ERR_ABORTED 404 (Not Found).

However, if I delete the original Docker image before re-building, then I don't get this error.

Similarly, when I try to deploy my app to Elastic Beanstalk, if I rebuild the environment after pushing a new React Docker image, it works, but if I just re-deploy, without re-building, I get the same error as above.

Am I missing something very obvious? It seems like maybe the locations of the static files are changing between builds, and if I don't completely get rid of the original image, it's still pointing to (non-existent) files.

Here is docker-compose.yml

services:
  django:
    build:
      context: ./backend
      dockerfile: Dockerfile
    volumes:
      - django_static_volume:/usr/src/app/static
      - ./backend/django_app:/usr/src/app/django_app
    expose:
      - 8000
    ports:
      - "8000:8000"
    env_file:
      - ./backend/.env
    command: gunicorn --reload mainapp.wsgi:application --bind 0.0.0.0:8000
  react:
    build:
      context: ./frontend
      dockerfile: Dockerfile
      args:
        - API_SERVER=http://127.0.0.1
    volumes:
      - react_static_volume:/usr/src/app/build/static
    expose:
      - 3000
    env_file:
      - .env
    command: serve -s build -l 3000
    depends_on:
      - django

  nginx:
    restart: always
    build: ./nginx
    volumes:
      - django_static_volume:/usr/src/app/django_files/static
      - react_static_volume:/usr/src/app/react_files/static
    ports:
      - 80:80
    depends_on:
      - react

volumes:
  django_static_volume:
  react_static_volume:

and here is the react app Dockerfile:

###########
# BUILDER #
###########

# pull official base image
FROM node:12.18.3-alpine3.9 as builder

# set work directory
WORKDIR /usr/src/app

# install dependencies and avoid `node-gyp rebuild` errors
COPY ./react_app/package.json .
RUN apk add --no-cache --virtual .gyp \
        python \
        make \
        g++ \
    && npm install \
    && apk del .gyp

# copy our react project
COPY ./react_app .

# perform npm build
ARG API_SERVER
ENV REACT_APP_API_SERVER=${API_SERVER}
RUN REACT_APP_API_SERVER=${API_SERVER} \ 
  npm run build

#########
# FINAL #
#########

# pull official base image
FROM node:12.18.3-alpine3.9 

# set work directory
WORKDIR /usr/src/app

# install serve - deployment static server suggested by official create-react-app
RUN npm install -g serve

# copy our build files from our builder stage
COPY --from=builder /usr/src/app/build ./build

here is the nginx Dockerfile:

FROM nginx:1.19.0-alpine

RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d

WORKDIR /usr/src/app

and here is nginx.conf

upstream django_backend {
    server django:8000;
}

upstream react_frontend {
    server react:3000;
}

server {

    listen 80;

    ###########
    # URL ROUTING #
    ###########

    location /admin {
        proxy_pass http://django_backend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location /api {
        proxy_pass http://django_backend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location /patients {
        proxy_pass http://django_backend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    ###########
    # STATIC FOLDER ROUTING #
    ###########

    location /static/admin/ {
        alias /usr/src/app/django_files/static/admin/;
    }

    location /static/rest_framework/ {
        alias /usr/src/app/django_files/static/rest_framework/;
    }

    location /static/ {
        alias /usr/src/app/react_files/static/;
    }

    location /media/ {
        alias /usr/src/app/media/;
    }

    ###########
    # URL ROUTING #
    ###########

    location / {
        proxy_pass http://react_frontend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }


}