Python: Why does pip install not deploy my code from sdist?

user_33404 0
Viewed 75

I am trying to package a small Python 3.9 app in a .tar.gz software distribution file (aka "sdist").

My code has this structure:

my-code
  PackageA
    PackageB
      Module1.py
      __init__.py 
    __init__.py
  setup.py

My setup.py file looks like this:

from setuptools import find_packages, setup

setup(
    name="my-code",
    version="0.1",
    install_requires=[],
    packages=find_packages(include=["PackageA.PackageB"]),
    include_package_data=True,
    description="My Code"
)

When I run python setup.py sdist --formats=gztar I successfully get a my-code-0.1.tar.gz file and I can see that it contains my .py files.

However, when I run pip install my-code-0.1.tar.gz it seems that pip does not compile and deploy my .py files. I.e. when I install into a venv and look inside the Lib/site-packages directory for the venv I only see a directory called my_code-0.1-py3.9.egg-info and my code is not in it. Attempts to import or run my module that's in PackageA.PackageB fail.

My question is - Why is my code not built and installed by pip install and how can I fix this?

pip setuptools python
1 Answers

TL;DR because find_packages(include=["PackageA.PackageB"]) will filter out the parent PackageA, so it is not included in the installation. Just use

setup(
    packages=find_packages(),
    ...
)

and it will be fine.

Longer explanation is that the include arg does not mean "include in addition to what find_packages() finds". It means "include only packages found by find_packages() which are in the include filter list", so it can only shrink the package selection. Compare the output of

$ python -c "from setuptools import find_packages as f; print(f())"
['PackageA', 'PackageA.PackageB']

vs

$ python -c "from setuptools import find_packages as f; print(f(include=['PackageA.PackageB']))"
['PackageA.PackageB']

Since PackageA is not included, PackageA/__init__.py will be omitted in the source distribution, effectively removing the package property from PackageA - in the tar archive, it will be a regular directory now. Running pip install mydist.tar.gz won't find PackageA anymore, thus PackageA.PackageB is not findable as well. Consequently, nothing will be installed. Automatic package discovery section in setuptools docs contains a short mentioning of the include and exclude arguments to find_packages(), but IMO a lot more helpful is the function's docstring:

>>> from setuptools import find_packages
>>> help(find_packages)
find(where='.', exclude=(), include=('*',)) method of builtins.type instance
    Return a list all Python packages found within directory 'where'

    ...

    'include' is a sequence of package names to include.  If it's
    specified, only the named packages will be included.  If it's not
    specified, all found packages will be included.  'include' can contain
    shell style wildcard patterns just like 'exclude'.
user_26502490
Related