I have a basic bash script which runs parallel jobs on compute node containing "X" cores. The script goes through a series of folders in a numeric order and execute code within:
#!/bin/bash
no_proc_per_calc=x
no_proc_to_use=X
ini=${v1}
fin=${v2}
counter=1
max_parallel_calcs=$(expr $no_proc_to_use / $no_proc_per_calc)
for ii in $(eval echo "{${v1}..${v2}}"); do
cd ${ii}
./execute &
echo $PWD
sleep 5
cd ../
mod_test=$(echo "$counter % $max_parallel_calcs" | bc)
if [ $mod_test -eq 0 ] ; then
wait
fi
counter=$(expr $counter + 1)
done
wait
In this script x is a divisor for both X and v2-v1 (and X/x is usually a divisor of v2-v1, but not necessarily) . The issue is that once the code chooses a block of X/x computations, it will stop until all computations in that block are over, which means if one of calculations takes longer time, X-x node cores will have to remain idle for that duration. Is there a way to continuously to do this type of block calculations, so if a single calculation finishes before the rest the script can move on the next calculation thereby running X/x calculations on the node for most of the computation time.
P.S.: As I am running this submit script on a job cluster, therefore I do not have the option to install new bash libraries. I would prefer if I can solve this problem on a standard out-of-the-box bash installation.