First off, let's clearly state what a duplicate library can mean in this context. If you were to unpack your bundled project, you might find multiple versions of React, for example, 16 and 17. This is a good framing to keep in mind for the rest of the discussion.
While most of the time, this kind of duplication is a mistake, there are cases where this may be desirable. Let's cover that first before getting to your specific case.
A good example of where this can be beneficial is big distributed development teams that use something similar to the micro frontends architecture. Shortly explained, two teams might be responsible for two different parts of the page with their own set of dependencies, without affecting the other:
┌───────────────────────────┐
│ Webpage │
│ ┌──────────┐ ┌──────────┐ │
│ │ React 16 │ │ React 17 │ │
│ └──────────┘ └──────────┘ │
└───────────────────────────┘
While this may seem like waste of space, the development team has made a tradeoff here: they have a slightly bigger page load, but they reduce the need for inter-team syncing regarding versions. Perhaps one team is ready to use the latest version, but the other still needs to do work before they're ready to update. Another good example is legacy code — you may have code that works well, but is unmaintained so you choose to just ship it as is as long as it works:
┌──────────────────────────────┐
│ Webpage │
│ ┌───────────┐ ┌────────────┐ │
│ │ Angular 1 │ │ Angular 10 │ │
│ └───────────┘ └────────────┘ │
└──────────────────────────────┘
Hopefully the above illustrates why setups like this are allowed by the package manager: they can be desirable, and in some cases, very beneficial.
The way this can be set up varies as widely as there are project setups, but one of the approaches to the above is having subprojects of some sort, each with their own package.json:
project
├─ package.json
├─ one subproject
│ └─ package.json
└─ another subproject
└─ package.json
In the above case, imports in one subproject would be set up to resolve by the package.json in that folder, and imports in another subproject would be set up to resolve by their package.json. One common way to configure module resolution is via the Webpack resolve configuration, but other approaches are of course possible.
Hopefully the above explains why this is possible, as well as some hints as to when it might be a good consideration.
As mentioned before though, most of the time this is a mistake. Most users don't intentionally want to do this, and including the same library with minute differences, for example React 17.0.0 and React 17.0.1 is not what they meant to do. For the common use case, the above two should be practically interchangeable.
There are a number of ways to end up with this issue, but the most common one is incorrectly configured peer dependencies. Peer dependencies are a way for libraries, component packs, etc to specify what they rely on, while allowing the dependency to be shared. A common example would be a UI component library that supports any React version as long as it's newer than, say, React 16. If your project uses React 17, you can then happily pull in that component library and have a single version of React in your code base.
The error occurs when the library author (unintentionally) configures the dependency on React 16 as a regular dependency, not a peer dependency. In that case, the author is essentially saying the library needs React 16 and doesn't allow anything else, and you would end up with two versions of React installed.
All of the above is very broad strokes, and brushing over a lot of details, but hopefully this gives you some insight into the problem you're solving.
