is there a way to trigger 10 scripts at any given time in Linux shell scripting?

user_7870508 0
Viewed 104

I have a requirement where I need to trigger 10 shell scripts at a time. I may have 200+ shell scripts to be executed.

e.g. if I trigger 10 jobs and two jobs completed, I need to trigger another 2 jobs which will make number of jobs currently executing to 10.

I need your help and suggestion to cater this requirement.

shell unix bash
4 Answers

Yes with GNU Parallel like this:

parallel -j 10 < ListOfJobs.txt

Or, if your jobs are called job_1.sh to job_200.sh:

parallel -j 10 job_{}.sh ::: {1..200}

Or. if your jobs are named with discontiguous, random names but are all shell scripts named with .sh suffix in one directory:

parallel -j 10 ::: *.sh

There is a very good overview here. There are lots of questions and answers on Stack Overflow here.

user_28366210

Simply run them as background jobs:

for i in {1..10}; { ./script.sh & }

Adding more jobs if less than 10 are running:

while true; do
    pids=($(jobs -pr))
    ((${#pids[@]}<10)) && ./script.sh &
done &> /dev/null
user_126074430

There are different ways to handle this:

  • Launch them together as background tasks (1)
  • Launch them in parallel (1)
  • Use the crontab (2)
  • Use at (3)

Explanations:

(1) You can launch the processes exactly when you like (by launching a command, click a button or whatever event you choose)
(2) The processes will be launched at the same time, every (working) day, periodically.
(3) You choose a time when the processes will be launched together once.

user_42791550

I have used below to trigger 10 jobs a time.

max_jobs_trigger=10



while mapfile -t -n ${max_jobs_trigger} ary && ((${#ary[@]})); do
    jobs_to_trigger=`printf '%s\n' "${ary[@]}"`

#Trigger script in background

done
user_78705080
Related