conflict using conda and pip in miniconda environment

user_305883 0
Viewed 3577

I found that if I try to install tqdm in miniconda environment (pip insall tqdm), it will fail, and the error then will persist also for other libraries searched by pip

Update

I tried many test, I come to the conclusion that conda and pip install conflicts in miniconda environment.

Step to reproduce:

conda create --name mlvision

test : is pandas and fastai there?

pip install pandas > pandas is there

now try to install opencv via conda

conda install opencv
[..]

now if I try pip install, I will get error:

pip install pandas
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/fc/43/fd867e3347559845c8f993059d410c50a1e18709f1c4d4b3b47323a06a37/pandas-0.24.2-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
dyld: lazy symbol binding failed: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/gg4u/Sites/miniconda3/envs/mlvision/bin/python
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/gg4u/Sites/miniconda3/envs/mlvision/bin/python
  Expected in: /usr/lib/libSystem.B.dylib

All installations via pip are now broken:

pip install fastai
Collecting fastai
  Using cached https://files.pythonhosted.org/packages/7d/14/385382da888f7e3ba24c726d95ea05563eb5e984ed97d5a24e74197f7848/fastai-1.0.54-py3-none-any.whl
dyld: lazy symbol binding failed: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/gg4u/Sites/miniconda3/envs/mlvision/bin/python
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/gg4u/Sites/miniconda3/envs/mlvision/bin/python
  Expected in: /usr/lib/libSystem.B.dylib

also:

pip install --upgrade pip
Requirement already up-to-date: pip in ./envs/mlvision/lib/python3.7/site-packages (19.1.1)
dyld: lazy symbol binding failed: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/gg4u/Sites/miniconda3/envs/mlvision/bin/python
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/gg4u/Sites/miniconda3/envs/mlvision/bin/python
  Expected in: /usr/lib/libSystem.B.dylib

Trace/BPT trap: 5

I conclude that conda install may conflict with pip install...

not sure if only when using conda install opencv

The way around I found is to install opencv as last package, once used I may have the missing reference:

Expected in: /usr/lib/libSystem.B.dylib

and so I should install all packages via pip first, and last opencv...

How to solve conflicts in local conda environments ?

Tried to look at https://github.com/ContinuumIO/anaconda-issues/issues/1429 but could not yet solve the issue.

I cannot proceed to install all the packages I need (some via conda, some via pip)

note python correctly poiting to local env: 

which python
[your path]/miniconda3/envs/mlvision/bin/python

conda --version conda 4.7.5

using Mac OS Mavericks 10.9.5 (if it matters)

tqdm fast-ai conda pip python
1 Answers

Don't mix pip with conda if you can help it. Get everything you can from conda first. Then use pip only to install packages that you cannot get from conda.

The packages conda installs are created with the Anaconda toolchain. And they have dependencies that make sure that those were also created with a compatible Anaconda toolchain. If you mix in stuff with pip, it can install compiled libraries that are not compatible with what the Anaconda packages require.

user_114515090
Related