I built a python package and uploaded to pypi. It can be installed but not imported: ModuleNotFoundError

user_9866659 0
Viewed 87

I built a python package and uploaded to pypi. It installs just fine, however on import I get ModuleNotFound error. The error was replicated in brand new conda environments, on different machines (Ubuntu, MacOS, Windows. Although all in (new) conda envs). The last time I had this error was because the folder was incorrectly named, however that is not the case here. GitHub repo on branch and in folder that was used to build the whl.

Any ideas?

Package was built and installed using

python3 -m build
python3 -m twine upload dist/*
pip install pillaralgos

(pillar_env) jupyter-pomkos@jupyterubuntu:~$ python
Python 3.8.8 (default, Feb 24 2021, 21:46:12) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
---------------------------------------------------------------------------
>>> import pillaralgos
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pillaralgos'
>>>

It shows up in pip list

(pillar_env) jupyter-pomkos@jupyterubuntu:~$ pip list
Package            Version
------------------ -------------------
# Truncated
pexpect            4.8.0
pickleshare        0.7.5
pillaralgos        1.0.1
Pillow             8.2.0
pip                21.0.1

sys.path does include the directory that the library is installed in.

(pillar_env) jupyter-pomkos@jupyterubuntu:~$ python
Python 3.8.8 (default, Feb 24 2021, 21:46:12) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/jupyter-pomkos/.conda/envs/pillar_env/lib/python38.zip', '/home/jupyter-pomkos/.conda/envs/pillar_env/lib/python3.8', '/home/jupyter-pomkos/.conda/envs/pillar_env/lib/python3.8/lib-dynload', '/home/jupyter-pomkos/.local/lib/python3.8/site-packages', '/home/jupyter-pomkos/.conda/envs/pillar_env/lib/python3.8/site-packages']
>>>

pip show says it was installed in the correct env and dir.

(pillar_env) jupyter-pomkos@jupyterubuntu:~$ pip show pillaralgos
---------------------------------------------------------------------------
Name: pillaralgos
Version: 1.0.1
Summary: Algorithms for Pillar. Currently includes "mini" algorithms, nothing too sophisticated.
Home-page: https://github.com/pillargg/twitch_chat_analysis/tree/pypi_reorganize
Author: Peter Gates
Author-email: pgate89@gmail.com
License: UNKNOWN
Location: /home/jupyter-pomkos/.conda/envs/pillar_env/lib/python3.8/site-packages
Requires: 
Required-by: 
Note: you may need to restart the kernel to use updated packages.

ModuleNotFoundError appears in jupyter console, in jupyterlab notebook, and in the terminal python. Confirmed that the kernel is pointing at the correct conda directory (although I don't think this troubleshooting was necessary, since the error was replicated on different machines):

(pillar_env) jupyter-pomkos@jupyterubuntu:~$ nano ~/.local/share/jupyter/kernels/pillar_env/kernel.json
---------------------------------------------------------------------------
{
 "argv": [
  "/home/jupyter-pomkos/.conda/envs/pillar_env/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Pillar Env",
 "language": "python"
}
python-packaging setuptools
2 Answers

Your current setup.cfg looks like this:

[options]
package_dir =
    = pillaralgos
packages = find:
# ...

[options.packages.find]
where = pillaralgos

Comparing this with your project directory structure, it seems to me like you should change to something like this:

[options]
packages = find:
# ...

If I am not mistaken, the package_dir field as well as the whole [options.packages.find] section are not necessary.

user_111382590

Switched to poetry for all package building/publishing, they seem to have their head on straight.

That said to run with python's inbuilt python3 -m build and then python twine upload dist/* commands:

  • Make sure all folders have __init__.py in them, locally and on github
  • Make sure the folder structure is correct <--- this was the issue

Folder structure for default python command should be:

|-- pypi
     |-- src  # <---- this folder can be named pillaralgos, but still needed another pillaralgos subfolder
          |-- pillaralgos  # <----- this is needed
               |-- helpers
                    |-- __init__.py
                    |-- data_handler.py
               |-- __init__.py
               |-- algoXX.py
     |-- LICENSE
     |-- pyproject.toml
     |-- README.md
     |-- setup.cfg

poetry worked out of the box. Folder structure:

|-- pypi
    |-- pillaralgos  # <---- note that poetry didn't require an additional subfolder
        |-- helpers
            |-- __init__.py
            |-- data_handler.py
            |-- graph_helpers.py
            |-- sanity_checks.py
        |-- __init__.py  # must include version number
        |-- algoXX.py  # all algorithms in separate files
    |-- LICENSE
    |-- README.md
    |-- pyproject.toml  # must include version number
user_98666590
Related