conda create from environment file vs. conda clone

user_3447047 0
Viewed 775

We're keeping old instances of conda environments around such that people can clone them.

I noticed that when I use conda create --clone, that the packages are still downloaded and not copied.

Therefore I am wondering whether just keeping an exported environment.yaml file has the same effect.

Basically, the question is, how are the two following cases different (if at all)?

conda activate old_env
conda env export > env.yml
conda create -n new_env -f env.yml

vs

conda create -n new_env --clone /path/to/old_env

I have looked at the conda source code here but I am not familiar enough with the code base to understand what is happening.

conda python
2 Answers

Don't have a comprehensive answer readily at hand, but two differences I notice are:

  • Cloning copies all untracked files. While conda-env will detect PyPI packages and include them in the YAML, conda create --clone will detect any non-Conda files and copy them. This could be useful if one is installing other software into the environment not through Conda or Pip. For example, an R env where one may have installed something from GitHub using devtools::install_github() rather than Conda. So cloning can sustain customized environments.
  • Minor differences in explicit specifications on new environment. This is a bit trivial, but cloning will include the sourced channel and subdir in the explicit specifications, whereas YAMLs will not (unless originally include). For example, running conda env export in the new environment will give something like numpy=1.20.1=py39h3c955ea_0 for environment created by YAML and conda-forge/osx-64::numpy==1.20.1=py39h3c955ea_0 for cloned environments. This is mostly inconsequential, since the build string itself (here py39h3c955ea_0) is almost always sufficiently unique to discern the channel of origin.

I'm unsure about why you see the redownloading. That is possibly sensitive to whether users have a shared package cache (pkgs_dirs) or side-effects from users cleaning the package cache (conda clean -tp). In a test case I just did, there were some redownloaded packages, but only a subset. Note that you can also try using the --offline flag to only use the package cache.

user_5709180

The difference is that you can't clone an environment across separate computers, but you can easily send a yaml file across the world and then replicate your environment on another machine.

However, conda is caching all downloaded packages in the pkgs folder and then hardlinking them (as far as possible) to the environments to save disk space. This is somewhat in contradiction to what you supposedly observed during the clone process. Probably it's just extracting while it says "Downloading and Extracting Packages".

user_71085890
Related