Static React Website on AWS (S3 + CloudFront) - return HTTP 404 when page is not found

I have a React website hosted statically on AWS S3 + CloudFront. I have a "Not found" configured in React when the request doesn't match any route:

<Switch>
  ...
  <Route path="/404.html" exact={true} component={PageNotFound}/>
  <Redirect to="/404.html" />
</Switch>

But, since the routing is client-side, the "not found" page still returns HTTP 200. I want this page to return HTTP 404 instead.

I could configure React router to redirect to a non-existent object in S3, but CloudFront is configured to overwrite 403's and 404's from S3 with 200's:

  CloudFrontDistribution:
    Type: AWS::CloudFront::Distribution
    Properties:
        ...
        CustomErrorResponses:
          - ErrorCachingMinTTL: 60
            ErrorCode: 404
            ResponseCode: 200
            ResponsePagePath: "/index.html"
          - ErrorCachingMinTTL: 60
            ErrorCode: 403
            ResponseCode: 200
            ResponsePagePath: "/index.html"

This seems to be the standard way to host SPAs on S3+CloudFront, e.g. https://gist.github.com/bradwestfall/b5b0e450015dbc9b4e56e5f398df48ff#spa.

How do I make my "Not found" page return HTTP 404 instead of 200?