Poetry install fails during docker build with CalledProcessError and EnvCommandError

Viewed 223

Problem summary

I've been trying to build a docker image which installs dependencies using poetry command RUN poetry install.
I am doing it as part of docker build job of the GitLab CI/CD pipeline, running it on the own runner VM instance.
However, docker build process repeatedly crashes during poetry install run with CalledProcessError, and then, during handling of this error, with EnvCommandError and PoetryException errors.
Few sample errors are below:

 CalledProcessError
 Command '['/home/ray/anaconda3/bin/python', '/home/ray/anaconda3/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.0.4-py3-none-any.whl/pip', 'install', '--disable-pip-version-check', '--prefix', '/home/ray/anaconda3', '--no-deps', '/home/ray/.cache/pypoetry/artifacts/a8/b9/10/d38e9304d486f9398ab4163054bd6af79c74073ad8685d6cc67f88eccc/Werkzeug-2.2.2-py3-none-any.whl']' returned non-zero exit status 1.

Then

EnvCommandError
Command ['/home/ray/anaconda3/bin/python', '/home/ray/anaconda3/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.0.4-py3-none-any.whl/pip', 'install', '--disable-pip-version-check', '--prefix', '/home/ray/anaconda3', '--no-deps', '/home/ray/.cache/pypoetry/artifacts/a8/b9/10/d38e9304d486f9398ab4163054bd6af79c74073ad8685d6cc67f88eccc/Werkzeug-2.2.2-py3-none-any.whl'] errored with the following return code 1, and output: 
WARNING: Ignoring invalid distribution -irtualenv (/home/ray/anaconda3/lib/python3.9/site-packages)
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/home/ray/anaconda3/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.0.4-py3-none-any.whl'

Then also PoetryException error:

PoetryException
Failed to install /home/ray/.cache/pypoetry/artifacts/a8/b9/10/d38e9304d486f9398ab4163054bd6af79c74073ad8685d6cc67f88eccc/Werkzeug-2.2.2-py3-none-any.whl
at anaconda3/lib/python3.9/site-packages/poetry/utils/pip.py:55 in pip_install

Same series of errors happen for Werkzeug-2.2.2 (listed above) and for torch-1.11.0 (not listed to spare space).

Attempts to resolve

I tried the following without success:

  • Cleaning PyPI cache as described here by adding command pip cache purge to my docker file or before building docker image.
  • I wanted to somehow follow the suggestion from Quick Fix described here. However, I cannot locate those artifacts /home/ray/.cache/pypoetry/artifacts/a8/b9/10/d38e9304d486f9398ab4163054bd6af79c74073ad8685d6cc67f88eccc/Werkzeug-2.2.2-py3-none-any.whl. So I don't understand how to delete them.

I can share the docker file, if needed.

I will appreciate any help!

1 Answers
RUN pip install -U pip virtualenv

Poetry gets and cache path to pip wheel on startup from virtualenv package with virtualenv.seed.wheels.embed.get_embed_wheel. Looks like virtualenv can be update while poetry install is running and this path become invalid.

Related