Job submission using "for loop"

Question

What I have been working on is submitting 173 jobs (lsf files) in a row. More specifically, I would like to submit the next job right after the previous job has been done.

What I am trying to follow is this method: (this script help the next step is started right after the previous Job has been done.)

bsub -J STEP1 < STEP1.lsf
bsub -J STEP2 -w 'done(STEP1)' < STEP2.lsf
bsub -J STEP3 -w 'done(STEP2)' < STEP3.lsf

problem Because there are more than 100 lsf files to execute in a row, I would like to use for loop method (or if there's any, I would love to accept that method)

Here is what I already wrote

source script

DIR="/sc/arion/projects/Itan_lab/minju/annotations/gnomAD/split/test"
NAME="t"

The shell script to execute several lsf files

#!/bin/bash
source ./pSTEP0.sh

for file in ${NAME}_*;do
    bsub < $DIR/${file}/"vep_${file}.lsf";
done

More detail I already have all the lsf files (vep_t_000.lsf ~ vep_t_172.lsf). The shell script works properly. However, due to memory issue I cannot run all of the 173 lsf files at the same time.

Question How can I write my code in a way that each lsf script is submitted right after the previous one has been completed.

vep_t_000.lsf (done) --> vep_t_001.lsf (submitted & start running) --> and so forth

Job submission using "for loop"

0 Answers