Typescript project references: Handle third-party dependencies of referenced project

I have a project with 3 directories client, api, and shared. The shared directory contains typescript types and definitions inside an engine folder that I would like to share with client and api. Additionally, the shared directory requires some third party dependencies as well (specified in it's package.json file).

A very basic sample repository describing this scenario can be found here, under the branch project-references. Essentially, server.js in api will import and call a function from the Entity.ts class which in turn, relies on an external library (mathjs).

Now I have been trying to use project references (rather unsuccessfully!) to build my api (I have not looked into the client yet!) so that the shared code will be available in a dist folder within the api for use.

What I have done so far:

1. I have added the following tsconfig.build-api.json file to shared directory. The outDir points to a dist folder inside api. The idea is to move the compiled shared files to the api's dist folder, maintaining the same directory structure in dist so that my imports won't break. Similarly, I plan on having another tsconfig file for the client.

{
  "compilerOptions": {
    "lib": ["es2015", "es2017", "dom"],
    "module": "commonjs",
    "target": "es6",
    "declaration": true,
    "declarationMap": true,
    "composite": true,
    "moduleResolution": "node",
    "outDir": "../api/dist/shared"
  },
  "include": ["engine"],
  "exclude": ["node_modules", "lib"],
  "references": []
}

2. I have updated the api's tsconfig.json file to reference the shared directory's tsconfig.build-api.json file

{
  "compilerOptions": {
    // some options
    "outDir": "dist/api/src",
  },
  "references": [
    { "path": "../shared/tsconfig.build-api.json" }
  ],
  // some more options
}

3. I run tsc --build . from inside the api directory which creates a dist folder with the compiled api and shared code in it. However, when I try to run this built code, I get an error stating that the third party dependencies of the shared code cannot be found.

How can I make sure the third party dependencies of my referenced project (shared) are resolved without any issues inside api?

Note: Without the third party dependencies in my shared code, I could get all of this working without project references by just using relative paths to do the required imports. Thiw was explained here. However, with the external dependencies, I still cannot figure out a way to do this. Research has led to things like monorepositories, learna, yarn workspaces etc etc. But I would like to think this can be solved without relying on external tooling.