You have a few options here.
Production Mode
Set the environment variable PUBLIC_PATH to /app/
or
As mentioned in the other answer, use homepage field in package.json
Development Mode
The config is more of hardcoded into the app. You need to eject the app first to make any edits.
Step 1
npm run eject
Step 2
In config/webpack.config.js, Find the below section (Somewhere around line 67 - 68)
const publicPath = isEnvProduction
? paths.servedPath
: isEnvDevelopment && '/';
and change to
const publicPath = isEnvProduction
? paths.servedPath
: isEnvDevelopment && '/app/';
Step 3
In config/webpackDevServer.config.js, find the below section (Somewhere around line 60 - 65)
// It is important to tell WebpackDevServer to use the same "root" path
// as we specified in the config. In development, we always serve from /.
publicPath: '/',
And change to
publicPath: '/app',
Step 4
npm start