How to completely remove a dependency and its references with a Webpack plugin

I am trying to write a Webpack plugin that finds and completely removes dependencies (anything imported with require or import) that follow a certain pattern.

Disclaimer, I know there is the IgnorePlugin but there are 2 reasons I am not using that:

1. I am not just removing the imports, I am memorizing what I have removed and will create a separate bundle with these dependencies
2. More importantly, I have tried to clone that plugin and memorize what I have deleted but that leads to errors. Let me explain

If I look at the source code of theIgnorePlugin it will remove them by returning null in the beforeResolve hooks. But that does not seem to signal Webpack to really ignore it, rather than to return a nullish reference. Because I am working inside Next.js and that leads to an error where Next cannot find the references to the things I have removed.

So, this is my code:

class NextCriticalWebpackPlugin {
  pluginName = this.constructor.name;
  requests = [];

  apply(compiler) {
    compiler.hooks.normalModuleFactory.tap(this.pluginName, factory => {
      factory.hooks.beforeResolve.tap(this.pluginName, result => this.checkIgnore(result));
    });
  }

  checkIgnore(result) {
    if (result && result.request && result.request.startsWith('next-critical/loader')) {
      this.requests.push(result.request);
      return null;
    }
    return result;
  }
}

module.exports = NextCriticalWebpackPlugin;

If I run Webpack, everything seems ok, but when I run my build, it will throw errors that it cannot find the modules that I had removed:

Cannot find module 'next-critical/loader!/Users/luke/Arbeit/Stroeer/Projekte/paper/apps/web/src/critrical/ads.ts'

So I am wondering how I can tell Webpack to remove dependencies and its imports from the bundle.