I am trying to create a conda environment that contains both tensorflow and pytorch. As a minimal example, I am running
conda env create -n test -f test-env.yaml
using the following test-env.yaml file:
channels:
- conda-forge
- defaults
- pyviz
dependencies:
- python>=3.7,<3.8
- tensorflow>=2.0.0,<2.4.0
- pytorch>=1.0.0,<2.0.0
name: my_environment
I get the following long error complaining about conflicts:
$ conda env create -n test -f test-env.yaml
Collecting package metadata (repodata.json): done
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package _libgcc_mutex conflicts for:
pytorch[version='>=1.0.0,<2.0.0'] -> _openmp_mutex[version='>=4.5'] -> _libgcc_mutex[version='*|0.1',build='conda_forge|main|main']
python[version='>=3.7,<3.8'] -> libgcc-ng[version='>=9.3.0'] -> _libgcc_mutex[version='*|0.1',build='conda_forge|main']
Package python conflicts for:
tensorflow[version='>=2.0.0,<2.4.0'] -> python[version='2.7.*|3.6.*|3.7.*|3.8.*']
python[version='>=3.7,<3.8']
tensorflow[version='>=2.0.0,<2.4.0'] -> tensorboard[version='>=2.3.0'] -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3|>=3.2|>=3.5|>=3.7,<3.8.0a0']
pytorch[version='>=1.0.0,<2.0.0'] -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0']
pytorch[version='>=1.0.0,<2.0.0'] -> cffi -> python[version='2.7.*|3.5.*|3.6.*|3.6.12|3.6.12|3.7.9|3.6.9|3.6.9|3.6.9|3.6.9|>=3.5,<3.6.0a0|3.4.*|3.9.*|>=3.5|3.8.*|3.7.*|>=3.7|>=3.6,<3.7|>=3',build='1_73_pypy|3_73_pypy|4_73_pypy|5_73_pypy|2_73_pypy|0_73_pypy']
Package setuptools conflicts for:
tensorflow[version='>=2.0.0,<2.4.0'] -> tensorboard[version='>=2.3.0'] -> setuptools[version='>=41.0.0|>=41.4']
python[version='>=3.7,<3.8'] -> pip -> setuptools
Package numpy conflicts for:
tensorflow[version='>=2.0.0,<2.4.0'] -> tensorboard[version='>=2.3.0'] -> numpy[version='>=1.12.0|>=1.16|>=1.16.6,<2.0a0|>=1.16.1']
pytorch[version='>=1.0.0,<2.0.0'] -> numpy[version='>=1.11.3,<2.0a0|>=1.16.6,<2.0a0|>=1.19.5,<2.0a0|>=1.19.4,<2.0a0|>=1.14.6,<2.0a0']
Package cudatoolkit conflicts for:
pytorch[version='>=1.0.0,<2.0.0'] -> cudnn[version='>=7.6,<8.0a0'] -> cudatoolkit[version='11.0|11.0.*|>=10.0,<10.1|>=10.1,<10.2|>=9.0,<9.1|>=9.2,<9.3|>=10.2,<10.3|11.2|11.2.*|11.1|11.1.*|9.0.*|8.0.*']
pytorch[version='>=1.0.0,<2.0.0'] -> cudatoolkit[version='10.0.*|10.0|10.0.*|10.1|10.1.*|10.2|10.2.*|9.2|9.2.*|>=10.1.243,<10.2.0a0|>=9.2,<9.3.0a0|>=10.0.130,<10.1.0a0|9.2.*|>=9.0,<9.1.0a0|>=8.0,<8.1.0a0']
Package _openmp_mutex conflicts for:
pytorch[version='>=1.0.0,<2.0.0'] -> _openmp_mutex[version='>=4.5']
pytorch[version='>=1.0.0,<2.0.0'] -> mkl[version='>=2020.4,<2021.0a0'] -> _openmp_mutex=[build=*_llvm]The following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.17=0
- pytorch[version='>=1.0.0,<2.0.0'] -> __glibc[version='>=2.17,<3.0.a0']
- pytorch[version='>=1.0.0,<2.0.0'] -> magma[version='>=2.5.4,<2.5.5.0a0'] -> __glibc[version='>=2.17']
Your installed version is: 2.17
Note that strict channel priority may have removed packages required for satisfiability.
If I set channel_priority to flexible instead of strict, it does work. However, I would really like this to work with strict channel priority if possible. Is there any way to make it work?
Here is the output of conda info:
$ conda info
active environment : base
active env location : /home/kgullikson/miniconda3
shell level : 1
user config file : /home/kgullikson/.condarc
populated config files : /home/kgullikson/.condarc
conda version : 4.9.2
conda-build version : 3.20.3
python version : 3.6.11.final.0
virtual packages : __cuda=11.2=0
__glibc=2.17=0
__unix=0=0
__archspec=1=x86_64
base environment : /home/kgullikson/miniconda3 (writable)
channel URLs : https://conda.anaconda.org/conda-forge/linux-64
https://conda.anaconda.org/conda-forge/noarch
https://conda.anaconda.org/pyviz/linux-64
https://conda.anaconda.org/pyviz/noarch
https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
https://conda.anaconda.org/kgullikson/linux-64
https://conda.anaconda.org/kgullikson/noarch
package cache : /home/kgullikson/miniconda3/pkgs
/home/kgullikson/.conda/pkgs
envs directories : /home/kgullikson/miniconda3/envs
/home/kgullikson/.conda/envs
platform : linux-64
user-agent : conda/4.9.2 requests/2.25.1 CPython/3.6.11 Linux/3.10.0-1160.11.1.el7.x86_64 centos/7.6.1810 glibc/2.17
UID:GID : 207402735:207400513
netrc file : None
offline mode : False
And my .condarc:
channels:
- conda-forge
- pyviz
- defaults
- kgullikson
conda-build:
root-dir: ~/conda-builds
skip_existing: true
pip_interop_enabled: false
channel_priority: strict
UPDATE:
Reading more closely through the documentation on conda, it seems like this is actually the expected behavior of strict channel priority. https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-channels.html
Namely:
With strict channel priority, packages in lower priority channels are not considered if a package with the same name appears in a higher priority channel
So the problem here is that there are in fact tensorflow packages in conda-forge, just none that satisfy my requirement of using tensorflow 2. Given that they are pushing people to use strict and that they plan to make it the default in conda 5, that seems problematic...