How to convert pip install to Poetry file?

Question

With hours of research, I still can't figure out a way to convert this pip install cmd to pyproject.toml file. I'm trying to install PyTorch.

pip install torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

This is what I got at the moment (Completely Wrong!)

[tool.poetry]
name = "poetry-test"
version = "0.1.0"
description = ""
authors = ["Your Name "]

[tool.poetry.dependencies]
python = "^3.8"
torch = "^1.7.0"
torchvision = "^0.8.1"
torchaudio = "^0.7.0"

[tool.poetry.dependencies.torch]
url = "https://download.pytorch.org/whl/torch_stable.html"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

user_962190 · Answer

If you use a url specifier in your dependency definition, it needs to point directly to the file that you want to install, and not an .html page:

[tool.poetry.dependencies]
python = "^3.8"
torch = {url = "https://download.pytorch.org/whl/cpu/torch-1.7.0%2Bcpu-cp38-cp38-win_amd64.whl"}
torchaudio = {url = "https://download.pytorch.org/whl/torchaudio-0.7.0-cp38-none-win_amd64.whl"}
torchvision = {url = "https://download.pytorch.org/whl/cpu/torchvision-0.8.1%2Bcpu-cp38-cp38-win_amd64.whl"}

That being said, your setup can't be installed with poetry right now because you're going to get a solver error:

[SolverProblemError]
Because torchaudio (0.7.0) depends on torch (1.7.0) which doesn't match any versions, torchaudio is forbidden.
So, because no versions of torchaudio match !=0.7.0
 and poetry-test depends on torchaudio (*), version solving failed.

While torch can be installed in isolation, something goes wrong while matching versions when installing torchaudio or torchvision. Maybe it's got to do with the fact that torch includes build info in their version in the wheel you're trying to install (i.e. 1.7.0+cpu), maybe it's because url-denoted dependencies can't be assigned a version in the pyproject.toml (which makes sense, it's not like poetry gets to resolve anything if we tell it "here is a tarball, install it!"), or maybe just an internal poetry bug.

You might have to stick with pip for now, given that it will not complain about solving a dependency tree and instead just starts installing. But ultimately, blame the torch maintainers, for 1) not uploading all their wheels to the global Python package index, and 2) not even setting up their own file server in a PEP-503 conform way.

Update:

I looked a bit more into the discussions, and it seems that a helpful someone has done torch's job of serving their wheels on a PEP-503 conform server. So for the time being, the following will actually work:

[tool.poetry]
name = "poetry-test"
version = "0.1.0"
description = ""
authors = ["Your Name "]

[[tool.poetry.source]]
name = "foo"
url = "https://vxlabs.com/pypi//simple/"

[tool.poetry.dependencies]
python = "^3.8"
torch = "1.7.0"
torchvision = "0.8.1"
torchaudio = "0.7.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

user_16292873 · Answer

Previous solution didnt work for me as mentioned in this poetry issue#2543. So what worked for me in the meantime was to upgrade to version 1.2(preview) which addresses that issue.

Install poetry 1.2

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python - --preview

Add below repo to your pyproject.toml as suggested here issue#4124, and you shouldnt get version mismatch.

[[tool.poetry.source]]
name = "torch_rep"
url = "https://eternalphane.github.io/pytorch-pypi"

After that, you can just proceed to install them poetry add torch =1.7.0+cpu poetry add torchvision=0.8.1+cpu

Alternatively, adding dependencies to pyproject.toml and doing poetry install should also work.

How to convert pip install to Poetry file?

2 Answers