Wait for bash background jobs in script to be finished

user_47573 0
Viewed 50058

To maximize CPU usage (I run things on a Debian Lenny in EC2) I have a simple script to launch jobs in parallel:

#!/bin/bash

for i in apache-200901*.log; do echo "Processing $i ..."; do_something_important; done &
for i in apache-200902*.log; do echo "Processing $i ..."; do_something_important; done &
for i in apache-200903*.log; do echo "Processing $i ..."; do_something_important; done &
for i in apache-200904*.log; do echo "Processing $i ..."; do_something_important; done &
...

I'm quite satisfied with this working solution; however, I couldn't figure out how to write further code to be executed only once ALL of the loops have been completed.

Is there a way to do this?

job-control scripting bash
5 Answers

There's a bash builtin command for that.

wait [n ...]
      Wait for each specified process and return its termination  sta‐
      tus.   Each  n  may be a process ID or a job specification; if a
      job spec is given, all processes  in  that  job’s  pipeline  are
      waited  for.  If n is not given, all currently active child pro‐
      cesses are waited for, and the return  status  is  zero.   If  n
      specifies  a  non-existent  process or job, the return status is
      127.  Otherwise, the return status is the  exit  status  of  the
      last process or job waited for.
user_75360

A minimal example with wait $(jobs -p):

  for i in {1..3}
  do
    (echo "process $i started" && sleep 5 && echo "process $i finished")&
  done  

  sleep 0.1 # For sequential output
  echo "Waiting for processes to finish" 
  wait $(jobs -p)
  echo "All processes finished"

Exemplary output:

process 1 started
process 2 started
process 3 started
Waiting for processes to finish
process 2 finished
process 1 finished
process 3 finished
All processes finished
user_18459760
Related