How to use node --experimental-modules with Typescript output

Might be the stupidest question ever, but I have a Node project that is using ES modules with --experimental-modules and Node 12.

Now I'm adding an inner package to the monorepo that's written in Typescript that's consumed by the main node app. I'm struggling with my Typescript build settings that produce something that will work with --experimental-modules.

Currently tsconfig.json:

{
  "include": [
    "src/**/*ts"
  ],
  "exclude": [
    "node_modules",
    "**/*.spec.ts"
  ],
  "compilerOptions": {
    "module": "esnext",
    "esModuleInterop": true,
    "target": "esnext",
    "moduleResolution": "node",
    "sourceMap": true,
    "outDir": "dist"
  },
  "lib": ["es2015"]
}

If I have index.ts that imports from a neighbour ts file:

import { schema } from './schema'

The built import statements is without any .js and this makes my node app choke:

(node:78972) ExperimentalWarning: The ESM module loader is experimental.
internal/modules/esm/default_resolve.js:79
  let url = moduleWrapResolve(specifier, parentURL);
            ^

Error: Cannot find module /Users/viktor/dev/projects/kb-frontend/packages/graph/dist/schema imported from /Users/viktor/dev/projects/kb-frontend/packages/graph/dist/index.js

The reason is that the import is without .js - patching that in my dist directory of the ts build it works.

I cannot simply change my ts module to commonjs since this will also change the way my main exports are working with my esm based main giving me other errors:

import { server as graphMiddleware } from '@kb-front/graph'
         ^^^^^^
SyntaxError: The requested module '@kb-front/graph' does not provide an export named 'server'

I would not want to add a lot of .default here and there since the entire setup I have is to avoid commonjs alltogether and just use js and typescript with esm. Hey, I want the new shiny stuff.

What am I missing?