Does installing a local wheel with extras work on Python 3.11?

Question

I'm having an issue where I have a wheel I just built, and I want to install it with a particular set of "extra" installs. The name of the module in question is "scriptconfig".

Unfortunately, I don't have a MWE right now, but if I can't figure this out quickly I'll make one.

On Python 3.10 and 3.11 I can do:

python -m build --wheel --outdir wheelhouse

And that builds my wheel and puts it in ./wheelhouse. When I ls I see it is there:

(pyenv3.11-dev) joncrall@toothbrush:~/code/scriptconfig$ ls wheelhouse/
scriptconfig-0.7.0-py3-none-any.whl

I can install it normally in both 3.10 and 3.11

 pip install scriptconfig -f wheelhouse/

In both cases I get: Successfully installed scriptconfig-0.7.0

But when I try to add extras:

pip install scriptconfig[tests-strict,runtime-strict,optional-strict] -f wheelhouse

Python 3.6 - 3.10 works fine:

(pyenv3.10.5) joncrall@toothbrush:~/code/scriptconfig$ pip install scriptconfig[tests-strict,runtime-strict,optional-strict] -f wheelhouse
Looking in links: wheelhouse
Processing ./wheelhouse/scriptconfig-0.7.0-py3-none-any.whl
Requirement already satisfied: pyyaml in /home/joncrall/.pyenv/versions/3.10.5/envs/pyenv3.10.5/lib/python3.10/site-packages (from scriptconfig[optional-strict,runtime-strict,tests-strict]) (5.4.1)

... 

Installing collected packages: scriptconfig
Successfully installed scriptconfig-0.7.0

but on 3.11rc2 I it ignores wheelhouse and just installed 0.6.5 from pypi.

(pyenv3.11-dev) joncrall@toothbrush:~/code/scriptconfig$ pip install scriptconfig[tests-strict,runtime-strict,optional-strict] -f wheelhouse
Looking in links: wheelhouse
Processing ./wheelhouse/scriptconfig-0.7.0-py3-none-any.whl
Requirement already satisfied: pyyaml in /home/joncrall/.pyenv/versions/3.11-dev/envs/pyenv3.11-dev/lib/python3.11/site-packages (from scriptconfig[optional-strict,runtime-strict,tests-strict]) (6.0)
Requirement already satisfied: ubelt in /home/joncrall/code/ubelt (from scriptconfig[optional-strict,runtime-strict,tests-strict]) (1.2.2)
Collecting scriptconfig[optional-strict,runtime-strict,tests-strict]
  Using cached scriptconfig-0.6.5-py3-none-any.whl (32 kB)
WARNING: scriptconfig 0.6.5 does not provide the extra 'optional-strict'
WARNING: scriptconfig 0.6.5 does not provide the extra 'runtime-strict'
WARNING: scriptconfig 0.6.5 does not provide the extra 'tests-strict'
Requirement already satisfied: six in /home/joncrall/.pyenv/versions/3.11-dev/envs/pyenv3.11-dev/lib/python3.11/site-packages (from scriptconfig[optional-strict,runtime-strict,tests-strict]) (1.16.0)
Installing collected packages: scriptconfig
Successfully installed scriptconfig-0.6.5

Is this a known issue or a bug? What's going on?

Erotemic · Answer

I figured out why this was happening.

A part of my requirements file looks like this:

numpy>=1.21.6    ;                            python_version >= '3.10'    # Python 3.10+
numpy>=1.19.3    ; python_version < '3.10' and python_version >= '3.9'    # Python 3.9
numpy>=1.19.2    ; python_version < '3.9' and python_version >= '3.8'    # Python 3.8
numpy>=1.14.5    ; python_version < '3.8' and python_version >= '3.7'    # Python 3.7
numpy>=1.12.0    ; python_version < '3.7' and python_version >= '3.6'    # Python 3.6
numpy>=1.11.1    ; python_version < '3.6' and python_version >= '3.5'    # Python 3.5
numpy>=1.11.1    ; python_version < '3.5' and python_version >= '3.4'    # Python 3.4
numpy>=1.11.1    ; python_version < '3.4' and python_version >= '2.7'    # Python 2.7

This is parsed into setup.py to specify the minimum version for install_requires. But when an extra-requires is used with "strict", I have logic to change all of the ">=" to "==" which pins the version to the minimum. This works fine for 3.6-3.10, but for 3.11 I do not have an explicit entry in there yet and it was pinning numpy to the same min version as 3.10 would have. That would be ok, but there is no 1.21.6 wheel for 3.11.

I'm not sure of the exact mechanism, but that seemed to cause something to happen in pips dependency resolution logic and it feel back to an older version of scriptconfig (because I didn't specify which scriptconfig version I wanted).

Adding a new entry for 3.11 seems to do the trick:

numpy>=1.23.2    ;                             python_version >= '3.11'  # Python 3.11
numpy>=1.21.6    ; python_version < '3.11' and python_version >= '3.10'  # Python 3.10
numpy>=1.19.3    ; python_version < '3.10' and python_version >= '3.9'   # Python 3.9
numpy>=1.19.2    ; python_version < '3.9'  and python_version >= '3.8'   # Python 3.8
numpy>=1.14.5    ; python_version < '3.8'  and python_version >= '3.7'   # Python 3.7
numpy>=1.12.0    ; python_version < '3.7'  and python_version >= '3.6'   # Python 3.6
numpy>=1.11.1    ; python_version < '3.6'  and python_version >= '3.5'   # Python 3.5
numpy>=1.11.1    ; python_version < '3.5'  and python_version >= '3.4'   # Python 3.4
numpy>=1.11.1    ; python_version < '3.4'  and python_version >= '2.7'   # Python 2.7

So the problem was dependency constraints. Although it would have been nicer to have a better error message.

Does installing a local wheel with extras work on Python 3.11?

1 Answers