How to make pip fail early when one of the requested requirements does not exist?

user_1866775 0
Viewed 90

Minimal example:

pip install tensorflow==2.9.1 non-existing==1.2.3

Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorflow==2.9.1
  Downloading tensorflow-2.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (511.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 511.7/511.7 MB 7.6 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement non-existing==1.2.3 (from versions: none)
ERROR: No matching distribution found for non-existing==1.2.3

So pip downloads the (rather huge) TensorFlow first, only to then tell me that non-existing does not exist.

Is there a way to make it fail earlier, i.e., print the error and quit before downloading?

pip python
2 Answers

I'm afraid there's no straightforward way of handling it. I ended up writing a simple bash script where I check the availability of packages using pip's index command:

check_packages_availability () {
  while IFS= read -r line || [ -n "$line" ]; do
      package_name="${line%%=*}"
      package_version="${line#*==}"

      if ! pip index versions $package_name | grep "$package_version"; then
        echo "package $line not found"
        exit -1
      fi
  done < requirements.txt
}

if ! check_packages_availability; then
  pip install -r requirements.txt
fi

This is a hacky solution but may work. For every package in requirements.txt this script tries to retrieve information about it and match the specified version. If everything's alright it starts installing them.

Or you can use poetry, it handles resolving dependencies for you, for example:

pyproject.toml

[tool.poetry]
name = "test_missing_packages"
version = "0.1.0"
description = ""
authors = ["funnydman"]

[tool.poetry.dependencies]
python = "^3.10"
tensorflow = "2.9.1"
non-existing = "1.2.3"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

At the resolving stage it throws exception without installing/downloading packages:

Updating dependencies
Resolving dependencies... (0.2s)

SolverProblemError
    
Because test-missing-packages depends on non-existing (1.2.3) which doesn't match any versions, version solving failed.
user_99267210

Pip install requirements in order they are requested, so unless it gets to the faulty one it is unaware of the problem. But something like --dry-run (feature people requested for ages) would solve your problem. Luckily, it finally is part of pip starting from v22.2. See related ticket and pull request.

user_12356980
Related