ERROR: Cannot uninstall 'ruamel-yaml' while creating docker image for azure ML ACI deployment

user_4669722 0
Viewed 9743

I am trying to deploy machine learning model in azure ACI but i am getting following error while creating a docker image

Pip subprocess error:
ERROR: Cannot uninstall 'ruamel-yaml'. It is a distutils installed project and thus we cannot 
accurately determine which files belong to it which would lead to only a partial uninstall.

Below is my yml file for pip dependencies

name: project_environment
dependencies:
# The python interpreter version.

# Currently Azure ML only supports 3.5.2 and later.


- pip:
  # Required packages for AzureML execution, history, and data preparation.
  - pandas
  - azureml-defaults
  - azureml-sdk
  - azureml-widgets
  - numpy
  - tensorflow-gpu
  - keras
  - azureml-defaults
  - torch==1.4.0
  - scikit-learn==0.22.2.post1

and if i use conda instead of pip then i am getting following error

Step 11/13 : RUN CONDA_ROOT_DIR=$(conda info --root) && if [ -n 
"$AZUREML_CONDA_ENVIRONMENT_PATH" ]; then conda env update -p 
"$AZUREML_CONDA_ENVIRONMENT_PATH" -f '/var/azureml-app/binary_2.yml'; else 
conda env update -n base -f '/var/azureml-app/binary_2.yml'; fi && conda 
clean -aqy && rm -rf /root/.cache/pip && rm -rf "$CONDA_ROOT_DIR/pkgs" && 
find "$CONDA_ROOT_DIR" -type d -name __pycache__ -exec rm -rf {} +
---> Running in 9e6eb7278bfc  
[91mUnable to install package for Conda.

Please double check and ensure you dependencies file has
the correct spelling.  You might also try installing the
conda-env-Conda package to see if provides the required
installer. 
[0mThe command '/bin/sh -c CONDA_ROOT_DIR=$(conda info --root) && if [ -n 
"$AZUREML_CONDA_ENVIRONMENT_PATH" ]; then conda env update -p 
"$AZUREML_CONDA_ENVIRONMENT_PATH" -f '/var/azureml-app/binary_2.yml'; else 
 conda env update -n base -f '/var/azureml-app/binary_2.yml'; fi && conda 
clean 
-aqy && rm -rf /root/.cache/pip && rm -rf "$CONDA_ROOT_DIR/pkgs" && find 
"$CONDA_ROOT_DIR" -type d -name __pycache__ -exec rm -rf {} +' returned a 
non- 
 zero code: 255
 2020/08/12 19:36:09 Container failed during run: acb_step_0. No retries 
 remaining.
 failed to run step ID: acb_step_0: exit status 255

**Can anyone please help me **

azure-machine-learning-service docker machine-learning azure python
3 Answers

This is a problem ever since pip version 10 (relevant discussion on the pip repo). That thread details a few solutions:

Manually delete the files from site-packages. In my case, to delete ruamel.yaml the command was rm -rf /path/to/anaconda3/lib/python3.7/site-packages/ruamel*

You can also downgrade pip to version <10, reinstall ruamel with the --disable-pip-version-check option, then reverse the downgrade with pip install --upgrade pip.

Update: here's a 1-liner that deletes the ruamel packages:

rm -rf $(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")/ruamel*

Breaking that down: everything in the $(...) is a one-liner to get your global site-packages dir (source). The rm -rf deletes the ruamel package directories recursively. The /ruamel* suffix targets the ruamel package.

I've encountered this same issue with other packages (boto, gmpy2, pycosat) and the solution was the same, except to replace the suffix.

user_42121580

maybe use "pip install --ignore-installed [package]"

user_134227580

I was using a miniconda3 installation and had the same problem. Following crypdick's answer, I had to do the following using conda in Powershell:

# List outdated packages
pip list -o

# Try to update with:
pip install -U ruamel-yaml
# get error-1
...

# try to uninstall with conda: 
conda update ruamel-yaml

# get error-2
# PackageNotInstalledError: Package is not installed in prefix.
#   prefix: C:\Users\<user>\miniconda3
#   package name: ruamel-yaml

# Remove index cache, lock files, unused cache packages, and tarballs.
conda clean -a -vv
conda deactivate

# Remove any files related to "ruamel-yaml":
# WARNING: Recursive removal, so don't fuck this up!
cd $env:CONDA_PREFIX
gci -r -fi "ruamel*" | ForEach-Object { rm -Recurse -Force $_.FullName }

# From now on, "conda" will not work until we re-install:
# Make sure your pip version & path is correct.
pip -V 
pip install ruamel_yaml

Now conda works again!

user_11476880
Related