Accessing the paths of a Snakemake workflow's conda environments in rule inputs/outputs *without hardcoding*?

user_15704972 0
Viewed 159

I've got a Snakemake workflow which requires two separate conda environments, and one rule potentially downloads files and saves them into directories in the conda environment's path if they're missing (if you're curious: primer schemes and medaka models being automatically downloaded in the ARTIC pipeline). To avoid issues when multiple instances of that rule are run simultaneously for the first time in a new environment, I've inserted other rules to check for the presence of these files and pre-download them if needed (by having them be prerequisites of the offending rule).

Currently, I'm handling this with checkfiles, but I've been wondering if there was a way to access the paths of the various conda environments Snakemake is using within the Snakefile, equivalent to $CONDA_PREFIX in the shell, so I could check for the files directly? (Finding the environment with --list-conda-envs and hardcoding it isn't really an option, as it's supposed to be portable.)

EDIT: @dariober's answer nudged me into working out a workaround for this case - working out that artic minion could take a path for the --medaka-model argument and downloading the model somewhere less irritating to work with - and that's solved it (thanks!), but I still wonder: is there something, for example somewhere in snakemake.conda, that one could use to get at a Snakemake-created conda environment's path without having to find and hardcode it in a case where there's no such workaround?

snakemake conda
1 Answers

one rule potentially downloads files and saves them into directories in the conda environment's path

In my opinion, saving files in the conda directory is a poor design choice. If possible I would avoid that. However, assuming you don't want or cannot do that, maybe you should have a rule to create the conda environment. In this way you can tell via $CONDA_PREFIX where the conda environment is. E.g.:

rule create_env:
    output:
        touch('conda_env.done'),
    shell:
        r"""
        conda create -n my-env ...
        """

rule download_files:
    input:
        'conda_env.done',
    output:
        # Some file under $CONDA_PREFIX for my-env
    shell:
        r"""
        conda activate my-env
        
        # Download files and put them under $CONDA_PREFIX
        ...
        """

By the way, snakemake creates the conda environment under the .snakemake/conda directory in the working directory. However, I think it's going to be messy to find out which conda environment directory is the one you need.

user_11144530
Related