pip install -e doesn't allow to import via package dir

Easy script to reproduce bug in the end of the question

This was also submitted as a bug on pip's github

For a pkg structured like this:

.
├── setup.py
└── src
    └── my_pkg
        ├── __init__.py
        └── main.py

and a setup.py like this:

from setuptools import setup, find_packages

setup(
    name='test-pkg',
    packages=find_packages('src'),
    package_dir={
        'my_pkg': 'src/my_pkg',
    },
)

creating a wheel and installing it allows me to import "my_pkg", but installing the package using pip install -e . doesn't. Why?

Installing via wheel allows for importing "my_pkg" while installing via "-e" only allows for importing the "src" which defeats the purpose, as "src" is not the package dir

python -c "import my_pkg"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'my_pkg'

pip version: 21.3.1 (latest)

To summarize:

Working:

1. pip install .
2. python setup.py bdist_wheel && pip install dist/*whl

Not working:

1. pip install -e .
2. python setup.py develop (which is what pip install -e runs, but I wanted to make sure nothing got lost in the transition)
3. pip install -e . and then editing the /lib/python3.6/site-packages/test-pkg.egg-link and adding "src" to the path

Oopsie...

From setuptools docs:

For the .egg-info format, however, the base location is the directory that contains the .egg-info, and thus it is the directory that must be added to sys.path to make the egg importable. (Note that this means that the “normal” installation of a package to a sys.path directory is sufficient to make it an “egg” if it has an .egg-info file or directory installed alongside of it.)

I'm not entirely sure this means I'm screwed, but it might. Is there any way to go around this?

Bug reproduce script:

#!/bin/bash
set -e

mkdir -p reproduce_pip_install_e_bug/src/my_pkg
cd reproduce_pip_install_e_bug
touch src/my_pkg/__init__.py

echo "
from setuptools import setup, find_packages

setup(
    name='test-pkg',
    packages=find_packages('src'),
    package_dir={
        'my_pkg': 'src/my_pkg',
    },
)
" > setup.py

virtualenv -p python3.6 good_venv > /dev/null && . good_venv/bin/activate && pip install . -vvv &> .good_installation.log && python -c "import my_pkg"
echo this was successful, however, next command will break
virtualenv -p python3.6 bad_venv > /dev/null && . bad_venv/bin/activate && pip install -e . -vvv &> .bad_installation.log && python -c "import my_pkg"