poetry install different package version based on extras

user_5379182 0
Viewed 1975

Using python-poetry, I would like to install different package versions based on the extras that I pass during the installation. E.g. I would like to

# when extra == 'a', install numpy == 1.20.0
$ poetry install -E a
# when extra == 'b', install numpy == 1.19.0
$ poetry install -E b

I tried it with the following toml file

[tool.poetry]
name = "demo-poetry"
version = "0.1.0"
description = ""
authors = ["tenticon"]

[tool.poetry.dependencies]
python = "^3.8"
numpy = [
    { version = "1.20.0", markers = "extra == 'a'", optional = true},
    { version = "1.19.0", markers = "extra == 'b'", optional = true}
]

[tool.poetry.extras]
a = [ "numpy" ]
b = [ "numpy" ]

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

but when I do $ poetry install -E a I get

  SolverProblemError

  Because demo-poetry depends on both numpy (1.20.0) and numpy (1.19.0), version solving failed.

My poetry version is 1.1.6

python-poetry python
1 Answers

Poetry extras are sets of packages (e.g. a = ["numpy", "scipy"]) that can be optionally installed together with the main dependencies (poetry install -E a). When installing or specifying poetry-built packages, the extras defined in the toml file can be activated as described in PEP-508 definition of extras. Thus, the dependencies required by the a extra could be installed using pip install demo-poetry[a] (just like any other pip extras).

It is indeed possible to use environment markers as install conditions for dependencies (see list of PEP-508 environment markers). However, at the time of writing the environment marker extra is not returned by the relevant function get_marker_env(), and therefore it is ignored as install constraint. The PEP-508 documentation also notes that the extra variable is special, and that there is no current specification for it.

Neither using extras nor groups (from the pre-release 1.2.0a2) was sufficient in order to achieve the expected results. In this regard, I think the comment from @finswimmer is correct: groups are not mutually exclusive, but poetry checks that the dependency resolution works in every case.

The closest I could get to an acceptable solution was by defining conditions based on the python version, or based on the platform. Note that these conditions should always be mutually exclusive, otherwise you will get an error.

For instance, if you have the following pyproject.toml:

[tool.poetry]
name = "demo-poetry"
version = "0.1.0"
description = ""
authors = ["vreyespue"]

[tool.poetry.dependencies]
numpy = [
    { version = "1.19.0", python = "~3.7"},
    { version = "1.20.0", python = "~3.9"}
]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

You could install different versions of numpy in different environments:

$ poetry env use 3.7
  Using virtualenv: /***/demo-poetry-***-py3.7

$ poetry install
  Installing numpy (1.19.0)

$ poetry env use 3.9
  Using virtualenv: /***/demo-poetry-***-py3.9

$ poetry install
  Installing numpy (1.20.0)
user_62616500
Related