Unable to create a conda environment from a text file

user_16959593 0
Viewed 526

I've been trying to create a conda environment from a text file using this command

conda env create --name tf2_gpu --file /Dedicated/jmichaelson-wdata/ebahl/mlab/thesis/cell_activity/neuronal-activity-score/conda/tf2_gpu_pkgs.txt

and it keeps giving me this error

Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound: 
  - conda-forge/linux-64::gxx_impl_linux-64==7.5.0=hdf63c60_6
  - conda-forge/linux-64::gcc_impl_linux-64==7.5.0=hd420e75_6
  - @explicit
  - conda-forge/linux-64::gfortran_impl_linux-64==7.5.0=hdf63c60_6

I thought it could be because I have another version of miniconda (4.10.3) than the version that this environment text file is from (4.8.0). So, I reinstalled miniconda with v 4.8.0 and same python used for that env text file (3.7.8). However, I still get the same error when trying to create this new environment.

Any thought what could be the issue here?

Update: I searched for one of the packages (gfortran_impl_linux-64) using the command conda search gfortran_impl_linux-64=7.5.0[build=hdf63c60_6], and couldn't find it. So I tried looking for the package without specifying the build, conda search gfortran_impl_linux-64, and got this

Loading channels: done
# Name                       Version           Build  Channel             
gfortran_impl_linux-64           5.4.0      hdf63c60_3  pkgs/main           
gfortran_impl_linux-64           7.2.0      hb3c8cce_2  pkgs/main           
gfortran_impl_linux-64           7.2.0      hdf63c60_3  pkgs/main           
gfortran_impl_linux-64           7.3.0      hdf63c60_0  pkgs/main           
gfortran_impl_linux-64           7.3.0      hdf63c60_1  pkgs/main           
gfortran_impl_linux-64           7.3.0      hdf63c60_2  conda-forge         
gfortran_impl_linux-64           7.3.0      hdf63c60_3  conda-forge         
gfortran_impl_linux-64           7.3.0      hdf63c60_4  conda-forge         
gfortran_impl_linux-64           7.3.0      hdf63c60_5  conda-forge         
gfortran_impl_linux-64           7.5.0     h1104b78_14  conda-forge         
gfortran_impl_linux-64           7.5.0     h1104b78_15  conda-forge         
gfortran_impl_linux-64           7.5.0     h1104b78_16  conda-forge         
gfortran_impl_linux-64           7.5.0     h56cb351_18  conda-forge         
gfortran_impl_linux-64           7.5.0     h56cb351_19  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_10  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_11  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_12  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_13  conda-forge         
gfortran_impl_linux-64           7.5.0     ha8c8e06_17  pkgs/main           
gfortran_impl_linux-64           7.5.0     hfca37b7_17  conda-forge         
gfortran_impl_linux-64           8.4.0     h5ed45b9_17  pkgs/main           
gfortran_impl_linux-64           8.4.0     h603fa6f_17  conda-forge         
gfortran_impl_linux-64           8.4.0     h863adf9_14  conda-forge         
gfortran_impl_linux-64           8.4.0     h863adf9_15  conda-forge         
gfortran_impl_linux-64           8.4.0     h863adf9_16  conda-forge         
gfortran_impl_linux-64           8.4.0     hd6a5828_18  conda-forge         
gfortran_impl_linux-64           8.4.0     hd6a5828_19  conda-forge         
gfortran_impl_linux-64           8.5.0     h7faea26_10  conda-forge         
gfortran_impl_linux-64           8.5.0     h7faea26_11  conda-forge         
gfortran_impl_linux-64           8.5.0      h7faea26_8  conda-forge         
gfortran_impl_linux-64           8.5.0      h7faea26_9  conda-forge         
gfortran_impl_linux-64           9.3.0     h2bb4189_17  conda-forge         
gfortran_impl_linux-64           9.3.0     h5abd6ed_17  pkgs/main           
gfortran_impl_linux-64           9.3.0     h64c220c_11  conda-forge         
gfortran_impl_linux-64           9.3.0     h64c220c_12  conda-forge         
gfortran_impl_linux-64           9.3.0     h64c220c_13  conda-forge         
gfortran_impl_linux-64           9.3.0     hc4a2995_18  conda-forge         
gfortran_impl_linux-64           9.3.0     hc4a2995_19  conda-forge         
gfortran_impl_linux-64           9.3.0     hde52e87_14  conda-forge         
gfortran_impl_linux-64           9.3.0     hde52e87_15  conda-forge         
gfortran_impl_linux-64           9.3.0     hde52e87_16  conda-forge         
gfortran_impl_linux-64           9.4.0     h0003116_10  conda-forge         
gfortran_impl_linux-64           9.4.0     h0003116_11  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_3  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_4  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_5  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_6  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_7  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_8  conda-forge         
gfortran_impl_linux-64           9.4.0      h0003116_9  conda-forge         
gfortran_impl_linux-64          10.3.0     h73f4979_10  conda-forge         
gfortran_impl_linux-64          10.3.0     h73f4979_11  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_3  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_4  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_5  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_6  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_7  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_8  conda-forge         
gfortran_impl_linux-64          10.3.0      h73f4979_9  conda-forge         
gfortran_impl_linux-64          11.1.0      hc0c744c_3  conda-forge         
gfortran_impl_linux-64          11.1.0      hc0c744c_4  conda-forge         
gfortran_impl_linux-64          11.1.0      hc0c744c_5  conda-forge         
gfortran_impl_linux-64          11.1.0      hc0c744c_6  conda-forge         
gfortran_impl_linux-64          11.1.0      hc0c744c_7  conda-forge         
gfortran_impl_linux-64          11.1.0      hc0c744c_8  conda-forge         
gfortran_impl_linux-64          11.2.0     h7a446d4_10  conda-forge         
gfortran_impl_linux-64          11.2.0     h7a446d4_11  conda-forge         
gfortran_impl_linux-64          11.2.0      h7a446d4_8  conda-forge         
gfortran_impl_linux-64          11.2.0      h7a446d4_9  conda-forge

So, it could find the package but not the specified build it's looking for.

miniconda conda
2 Answers

I exported the environment file again without adding builds, and it worked. just added --no-builds to the export command.

user_169595930

The behavior here is due to packages being marked as broken. The Conda or Miniconda versions do not affect this.

Anaconda Cloud excludes packages labelled "broken"

Sometimes problems are discovered with specific builds of a package that can merit effective removal from search. Conda Forge documents how maintainers can trigger such a removal, and appears to be accomplished by adding the broken label to a package. Unfortunately, I can't find any specific Anaconda Cloud documentation on how this works, but broken is part of the default set of labels that an Anaconda repository expects.

In this specific case, one can see on the Anaconda Cloud website that the specific build is still available, but has been labelled broken:

anaconda cloud file tab for gfortran_impl_linux-64=7.5.0

As shown by OP, using default conda search settings does not report these:

Default search results

$ mamba search -q conda-forge::gfortran_impl_linux-64=7.5.0
Loading channels: ...working... done
# Name                       Version           Build  Channel             
gfortran_impl_linux-64           7.5.0     h1104b78_14  conda-forge         
gfortran_impl_linux-64           7.5.0     h1104b78_15  conda-forge         
gfortran_impl_linux-64           7.5.0     h1104b78_16  conda-forge         
gfortran_impl_linux-64           7.5.0     h56cb351_18  conda-forge         
gfortran_impl_linux-64           7.5.0     h56cb351_19  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_10  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_11  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_12  conda-forge         
gfortran_impl_linux-64           7.5.0     h64c220c_13  conda-forge         
gfortran_impl_linux-64           7.5.0     hfca37b7_17  conda-forge

However, the MatchSpec grammar is expressive enough to search under specific labels:

Searching the broken label

$ mamba search -q conda-forge/label/broken::gfortran_impl_linux-64=7.5.0
Loading channels: ...working... done
# Name                       Version           Build  Channel             
gfortran_impl_linux-64           7.5.0      h64c220c_7  conda-forge/label/broken
gfortran_impl_linux-64           7.5.0      h64c220c_8  conda-forge/label/broken
gfortran_impl_linux-64           7.5.0      h64c220c_9  conda-forge/label/broken
gfortran_impl_linux-64           7.5.0      hdf63c60_6  conda-forge/label/broken

or, since the main label is still retained on packages marked as broken, one could ignore the broken label by specifying to search all main packages:

Search all main packages

$ mamba search -q conda-forge/label/main::gfortran_impl_linux-64=7.5.0
Loading channels: ...working... done
# Name                       Version           Build  Channel             
gfortran_impl_linux-64           7.5.0     h1104b78_14  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h1104b78_15  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h1104b78_16  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h56cb351_18  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h56cb351_19  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h64c220c_10  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h64c220c_11  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h64c220c_12  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     h64c220c_13  conda-forge/label/main
gfortran_impl_linux-64           7.5.0      h64c220c_7  conda-forge/label/main
gfortran_impl_linux-64           7.5.0      h64c220c_8  conda-forge/label/main
gfortran_impl_linux-64           7.5.0      h64c220c_9  conda-forge/label/main
gfortran_impl_linux-64           7.5.0      hdf63c60_6  conda-forge/label/main
gfortran_impl_linux-64           7.5.0     hfca37b7_17  conda-forge/label/main

Recommended Workaround

As found by OP, a simple workaround is to remove the build part of the string (=hdf63c60_6) in the environment/package specification file. This enables Conda to substitute other builds that are not labelled as broken.

This can be done for all packages by including the --no-builds flag.

Workaround: Exact Environment Recreation

⚠️ Warning: The following approach enables packages labelled as broken to be installed! This is advanced usage that should only be performed by users who understand the consequences.

This last search result suggests that one can override the broken label behavior by including the channel conda-forge/label/main during installation or in a YAML file. There may be circumstances where this is desirable, such as needing to exactly recreate an environment for reproducing scientific results.

I would only recommend this for recreating an environment from a YAML or other environment serialization that includes build specifications, otherwise one might risk including broken packages in more than just the specific case. In practice, I would likely use a flexible channel priority, and keep the main label at lower priority. For example,

CONDA_CHANNEL_PRIORITY=flexible conda env create -n foo -c conda-forge -c conda-forge/label/main -f env.yaml

This helps ensure that non-broken packages still take priority if an equivalent case is encountered.

user_5709180
Related