Webpack 5 in Ceate React App can't resolve not fully specified routes

We are developing a react library and recently noticed, that it throws an error when we want to consume it with a new (webpack 5) Create React App. It complains about this:

"BREAKING CHANGE: The request failed to resolve only because it was resolved as fully specified (probably because the origin is strict EcmaScript Module, e. g. a module with javascript mimetype, a '.mjs' file, or a '.js' file where the package.json contains '"type": "module"')."

I managed to fix this, by adding

{
    test: /\.m?js/,
    resolve: {
      fullySpecified: false,
    },
}

While this works fine for any applications that have access to the webpack config, we'd like to make our lib "plug&play", without any eject or additional setup. What config should we have to make the consumer's webpack resolve our not fully specified routes? Code on github

Thanks in advance