Using non-fully-specified imports with nextjs and webpack 5

I am trying to use a library (a node module I don't control) that contains this import in the packaged code

import _omit from 'lodash/omit';

and it does not work with nextjs12 and webpack5. I get this error

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data .Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/dev/project/node_modules/lodash/omit' imported from /Users/dev/project/node_modules/library/es/library.js
Did you mean to import lodash/omit.js?

The library has "type": "module" in it, which I think is the cause of this?

I have seen that you need to add fullySpecifed: false, to the webpack loaders, and I have tried that a few different ways.

In my next config, i have

experimental: { esmExternals: true, fullySpecified: false },

I also tried doing this to the config in the webpack function.

config.module.rules.push({
  test: /\.m?js$/,
  type: 'javascript/auto',
  resolve: {
    fullySpecified: false,
  },
});

Is there something i am missing to make this work? It seems like it might have to do with the fact that this is during "collecting page data" and not "build"?

Node version: 14.16.0 yarn 1