i have a pipeline defined in a sub-workflow:
workflow pipeline{
main:
task1()
task2(task1.out)
...
}
that gets called by the main-workflow for different inputs:
workflow{
params.in = "$baseDir/input"
input_ch = Channel.fromPath(params.in + "/*.*")
main:
pipeline(input_ch)
}
its easy to add a label to process to define its resources. what i want to do is add such a label to my (whole) subworkflow. they label is supposed to force the subworkflow to execute as a if it was a single process (meaning that it does not invoke a new slurm call for all processes but only per subworkflow) like:
- pipeline for input 1 on node 1 as one slurmjob
- pipeline for input 2 on node 2 as one slurmjob
- main: - collect all the outputs
...
- pipeline for input n on node n as one slurmjob
i did as above by calling nextflow inside of nextflow:
process nextflowofnextflows {
label 'use_whatever_is_needed'
input:
file input1
output:
path("$out/*.*", emit: output)
script:
"""
nextflow run pipline.nf --in=${input1}
"""
}
... - but this is super unclean and i´d like to to it in a clean way.
whats the best practice to achieve the described behavior?
best, t.