Workflow with Snakemake & Conda: Placeholder of length '80' too short in package

user_4901684 0
Viewed 52

I have followed the Snakemake best practices here to create a workflow, where different steps activate different Conda environments. For example, a rule that collects statistics:

rule per_samp_stats: 
  input:
    ref_path='/labs/jandr/walter/tb/data/refs/{ref}.fa',
    bam='results/{batch}/{samp}/bams/{samp}_{mapper}_{ref}_sorted.bam'
  log:
    'results/{batch}/{samp}/stats/{samp}_{mapper}_{ref}_cov_stats.log'
  conda: 'envs/picard.yaml'
  params:
    prefix='results/{batch}/{samp}/stats/{samp}'
  shell:    
    '''
    {config[scripts_dir]}cov_stats.sh {input.ref_path} 
    '''

I am running into the error:

# CreateCondaEnvironmentException:
# Could not create conda environment from /oak/stanford/scg/lab_xx/xx/tb/mtb_tgen/workflow/envs/mtb.yaml:
# Command:
# mamba env create --quiet --file "/oak/stanford/scg/lab_xx/xx/tb/mtb_tgen/.snakemake/conda/5b3e765eb8210c23d169553fd7853814.yaml" --prefix "/oak/stanford/scg/lab_xx/xx/tb/mtb_tgen/.snakemake/conda/5b3e765eb8210c23d169553fd7853814"
# Output:
# Preparing transaction: ...working... done
# Verifying transaction: ...working... failed
# 
# PaddingError: Placeholder of length '80' too short in package /oak/stanford/scg/lab_xx/xx/tb/mtb_tgen/.snakemake/conda/5b3e765eb8210c23d169553fd7853814/bin/Rscript.
# The package must be rebuilt with conda-build > 2.0.

I'm wondering if you have suggestions how to deal with this; the problem seems to occur because of the long full path to the location where Snakemake rebuilds the environment to run a specific rule. Any help would be fantastic. Thank you!

mamba snakemake conda path python
1 Answers

This is only a hack, but one option is to use the named environments (which is not ideal, but should work). Basic steps are:

  1. create the environments manually (or have a rule that does it based on env files). For a rule-based approach it would be something like this:
rule create_env:
   input: 'env/{some_env}.yml'
   shell: 'conda env create -f {input}'

ENVS, _ = glob_wildcards('env/{env}.yml')

rule all:
    input: expand('env/{env}.yml',env=ENVS)
    output: touch('environments_created.txt')

The output file is to avoid running this rule all the time. There might be a better way to handle this.

2/ change conda directive to use the name of the env rather than a path. For example, if you create environment called 'custom_env123' then the rule definition will just include this name rather than path to the definition:

rule per_samp_stats:
   conda: 'custom_env123'  # name, not path
user_106935960
Related