I've written a snakemake workflow and to make code more manageable I split the rules into separate smk files. My current directory structure looks like this:
|-- Snakefile # with statements like include: utils/align.smk
|-- config.yaml
|-- envs
| `-- vep.yml
`-- utils
|-- align.smk
|-- envs
| |-- alignment.yml
| |-- mutect2.yml
| |-- quality_check.yml
| `-- vep.yml
|-- process_reads.smk
`-- variant_calling.smk
I also wrote the workflow to rely on conda and specified all the requirements to separate yaml files. Now, the location of the files is specified in config.yaml, for example:
CONDA_ALIGNMENT: "utils/envs/alignment.yml"
and then rules in align.smk call those by conda: config['CONDA_ALIGNMENT'].
However, now that I try to run snakemake, I get the following errror:
snakemake -np --profile cluster --jobs 2 --use-conda --force
Building DAG of jobs...
WorkflowError:
Failed to open source file /path/to/exec/utils/utils/envs/alignment.yml
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/exec/utils/utils/envs/alignment.yml'
File "/path/to/miniconda3/envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 228, in create
File "/path/to/miniconda3/envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 98, in hash
File "/path/to/miniconda3/envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 80, in content
File "/path/to/miniconda3/envs/snakemake/lib/python3.9/site-packages/snakemake/deployment/conda.py", line 67, in _get_content
Note that utils directory appears twice in the path for the file snakemake cannot find. All the other files in inputs/params/outputs... work when specified relative to the main Snakefile, rather than the particular smk file. I would be very grateful for any suggestions on how to deal with this in more snakemake way.
The fastest fix I can come up with would be to move the smk files out of utils but then it would be quite messy...