Is poetry ignoring extras or pyproject.toml is misconfigured?

Question

I have new project yolo created by poetry.

I did followed steps:

poetry new
poetry add requests
poetry add -D pytz
poetry add -D --optional --extras=dev ipdb
poetry lock

My toml file looks as follow:

[tool.poetry]
name = "yolo"
version = "0.1.0"
description = ""
authors = ["Your Name "]

[tool.poetry.dependencies]
python = "^3.6"
requests = "^2.24.0"

[tool.poetry.dev-dependencies]
pytest = "^5.2"
pytz = "^2020.1"
ipdb = {version = "^0.13.3", optional = true, extras = ["dev"]}

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

I removed the environment:

$ poetry env list
yolo-_0wi_Pw3-py3.6 (Activated)
$ poetry env remove yolo-_0wi_Pw3-py3.6
Deleted virtualenv: .cache/pypoetry/virtualenvs/yolo-_0wi_Pw3-py3.6

Now if I try to do:

$ poetry install
Creating virtualenv yolo-_0wi_Pw3-py3.6 in .cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 17 installs, 0 updates, 0 removals

  - Installing six (1.15.0)
  - Installing wcwidth (0.2.5)
  - Installing zipp (3.2.0)
  - Installing importlib-metadata (2.0.0)
  - Installing pyparsing (2.4.7)
  - Installing attrs (20.2.0)
  - Installing certifi (2020.6.20)
  - Installing chardet (3.0.4)
  - Installing idna (2.10)
  - Installing more-itertools (8.5.0)
  - Installing packaging (20.4)
  - Installing pluggy (0.13.1)
  - Installing py (1.9.0)
  - Installing urllib3 (1.25.10)
  - Installing pytest (5.4.3)
  - Installing pytz (2020.1)
  - Installing requests (2.24.0)
  - Installing yolo (0.1.0)

No ipdb, as expected.

But if I try:

$ poetry install --extras='dev'
Installing dependencies from lock file

[ValueError]
Extra [dev] is not specified.

To sort of clarify and explain more.

Toml file generated in my question is automatic work of poetry and there is a fix that requires manual intervention.

[tool.poetry]
name = "yolo"
version = "0.1.0"
description = ""
authors = ["Your Name "]

[tool.poetry.dependencies]
python = "^3.6"
requests = "^2.24.0"

[tool.poetry.dev-dependencies]
pytest = "^5.2"
pytz = "^2020.1"
ipdb = {version = "^0.13.3", optional = true, extras = ["dev"]}

[tool.poetry.extras]
dev = ["ipdb"]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Needless to say, it's confusing as hell.

But from now on if we do:

$ poetry install -E dev

It will work as expected and ipdb will be installed.

user_237258 · Answer

Spending some time me and my friend we figured out what is going on.

When you do:

poetry install --extra=dev ipdb

What in fact happens is that you specify you want ipdb that should use extra 'dev' that ipdb may or may not use.

Hence in toml it will be declared as:

[tool.poetry.dev-dependencies]
    ipdb = {version = "^0.13.3", extras = ["dev"]}

What in fact I want to achieve is to specify that extra for yolo project exists, it's called dev and includes installing ipdb. This is achieved by adding new section to poetry:

[tool.poetry.extras]
    dev = ["ipdb"]

The confusing factor is that both use keyword extra, while context is completely different. And main package extra is also in a different style than extra as part of the dependency definition.

Is poetry ignoring extras or pyproject.toml is misconfigured?

1 Answers