How can I transpile my TS files to ESM while making sure that all imports have their extensions?

I have a setup where I am transpiling Typescript files to ESM format using Babel. Here is my config:

{
  "presets": [
    [
      "@babel/preset-env",
      {
        "targets": {
          "esmodules": true
        },
        "modules": false
      }
    ],
    "@babel/preset-react",
    "@babel/preset-typescript"
  ]
}

When I check the output generated by Babel, I see that none of the imports have .js file extension in them:

// should be './SomeComponent.js' according to ESM spec
import SomeComponent from './SomeComponent';

Is there a way to enable it in Babel to make them ESM compliant?