pip install a project without copying the entire project folder

user_599739 0
Viewed 662

When I run pip install . on a project I'm working on, the entire project folder is copied to a temporary folder before pip starts the installation process. For me, this is a problem, because my project folder contains gigabytes of data that won't fit on my disk a second time. Therefore, pip install will fill up my disk and then crash when it runs out of free space.

This is not a problem when I run python setup.py install, because I clearly defined which folders should be part of my module and of course all that data is not included. Also, all that data is excluded in .gitignore, .dockerignore and whatnot. However, pip doesn't care about that, as the documentation states:

During regular installation, pip will copy the entire project directory to a temporary location and install from there. The exception is that pip will exclude .tox and .nox directories present in the top level of the project from being copied.

How can I prevent pip from copying my entire project folder and make it respect the contents of my carefully written setup.py file?

Background: I used to be happy with setup.py install, but recently I wanted to try git pre-commit, which internally calls pip install . to build the project. I didn't use pip as a build tool for my projects so far, but I thought this might be a good opportunity to at least make my project compatible with pip, but so far to no avail...

pip setup.py python
1 Answers

This seems to be a long standing issue in pip, with discussions since at least 2014 (see issue #2195).

There were a few proposals to solve this, like adding support for pipignore files that work like gitignore, but so far the only thing that was implemented is adding a hardcoded rule to ignore .tox and .nox folders, which sure solves the specific issues that these tools have, but not the problem in general.

A workaround is to create a source distribution (sdist) first and then to pip install that source package:

python setup.py sdist
pip install dist/<name>.tar.gz

Of course, this will not solve the issue with third party tools that call pip install . themselves. It has been discussed to make this the default behaviour of pip, but for reasons that are beyond me to understand this hasn't been touched in the last 6 years, so I don't expect a solution to this issue in the near future.

user_5997390
Related