ReactJS alias using tsconfig does not work

I have a ReactJS project and created one using create-react-app. I am trying to use aliases to avoid having too many '../../..'. I amended my tsconfig.json like so.

{
  ...
  baseUrl: ".",
  paths: {
    "@src/*": ["src/*"]
  }
  ...
}

I have a simple constants.ts file at /src/constants.ts

I am importing a constant in this file from /src/pages/landing/sign-in.tsx like so

import { CONST_VAL } from '@src/constants';

Typescript was able to detect my constants without issue. But when I try to run it, There was error saying "Unable to resolve module". Any idea why?

Tried this same technique in nextJS project and worked fine. But seems to have issue in a pure ReactJS app.