.htaccess file access for React Router

I have a situation where I can navigate through my page (start at localhost then click to redirect to a new page for example localhost/products), but when I refresh the localhost/products page I get a server 404 error. I am using React Router ("react-router-dom": "^6.2.2")

<Router>
...
  <Routes>
    <Route path="/" element={<Home />} />
    <Route path="/products" element={<Products products={products} />} />
    <Route path="/cart" element={<Cart />} />
    <Route path="/products/:id" element={<ProductDetail />}> </Route>
  </Routes>
</Router>

Here are two research links I found that address this problem
React-router urls don't work when refreshing or writing manually
https://akashmittal.com/react-router-urls-not-work-refresh-manua/

Both mention using a .htaccess file, I tried making one on my project folder and pasted this

<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /
  RewriteRule ^index\.html$ - [L]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_FILENAME} !-l
  RewriteRule . /index.html [L]
</IfModule>

It doesn't seem to work, I'm not sure if my webpack/project is accessing it.