How to add optional dependencies of a library as 'extra' in poetry and pyproject.toml?

user_9273406 0
Viewed 1028

I am building a Python package using pyproject and poetry. My pyproject.toml looks like this:

[tool.poetry]
authors = ["test"]
description = ""
name = "test"
version = "0.1.0"

[tool.poetry.dependencies]
spacy = {extras = ["cuda113"], version = "^3.2.3"}
faiss-gpu = {version = "1.7.2", optional = true}

[tool.poetry.extras]
gpu = ["faiss-gpu"]

This successfully installs faiss-gpu as an extra using poetry install -E gpu.

However, I would like to install spacy[cuda113] (GPU version) only when poetry install -E gpu is provided. A normal poetry install should only install spacy (CPU version).

I've tried using the following configuration, but this makes all of spacy optional and doesn't install it. Only spacy[cuda113] (GPU version) must be optional.

[tool.poetry]
authors = ["test"]
description = ""
name = "test"
version = "0.1.0"

[tool.poetry.dependencies]
spacy = {extras = ["cuda113"], version = "^3.2.3", optional = true}
faiss-gpu = {version = "1.7.2", optional = true}

[tool.poetry.extras]
gpu = ["faiss-gpu", "spacy"]

Is there a way to make spacy[cuda113] optional but spacy as a required dependency?

pyproject.toml python-poetry spacy pip python
1 Answers

Try like this:

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
authors = ["test"]
description = ""
name = "test"
version = "0.1.0"

[tool.poetry.dependencies]
python = "^3.10"
faiss-gpu = {version = "1.7.2", optional = true}
spacy = {extras = ["cuda113"], version = "^3.2.3", optional = true}
Spacy = "^3.2.3"

[tool.poetry.extras]
gpu = ["faiss-gpu", "spacy"]

It generates the dist-info METADATA lines like this, which looks correct in terms of how pip will install test vs test[gpu]:

Metadata-Version: 2.1
Name: test
Version: 0.1.0
Summary: 
Author: test
Requires-Python: >=3.10,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Provides-Extra: gpu
Requires-Dist: Spacy (>=3.2.3,<4.0.0)
Requires-Dist: faiss-gpu (==1.7.2); extra == "gpu"
Requires-Dist: spacy[cuda113] (>=3.2.3,<4.0.0); extra == "gpu"

Disclaimer: I think I am relying on bugs and/or implementation detail in poetry for this to work. Note the case sensitivity difference between "spacy" and "Spacy" in the dependency specification. The ordering of the lines is also crucial.

user_6740390
Related