How to track completion and errors of multiple parallel shell commands?

user_10172569 0
Viewed 51

How can I track the completion state and store errors (well, the entire log output) of multiple shell commands running in parallel?

I want to run 3 programs in parallel but I don't want to see their console logs while they are running. I want to instead display a list of all the programs and their exit status UPDATING IN REAL TIME.

# Script is started, no programs have completed
prog1 - Running...
prog2 - Running...
prog3 - Running...

# After some time passes, some programs will have completed
prog1 - Running...
prog2 - FAIL # exit code not 0
prog3 - PASS # exit code 0

Then, after all programs have finished running, I want to display the console logs of any failed program.

prog1 - FAIL
prog2 - FAIL 
prog3 - PASS
--- DONE ---
prog1 encountered an error. LOG:
# Then the entire prog1 log is printed. You would repeat this for every failed program

I know I can run programs in parallel with &

prog1 &
prog2 &
prog3

And I know I can send the stdout of a program to a temp file or something

prog1 > tempfile.txt

But I can't figure out how to track the process exit codes in realtime. I think I can do something with wait and then display only program logs that failed but I don't know how to store the completion status as a variable. I know $? gives you the exit code of the program that was LAST RUN but I want all of them, not just the last run. I need to be able to loop over all the result and then only print the temp logs of the failed programs. Any ideas?

parallel-processing shell bash
1 Answers

Without writing and testing all the code, I would do something like this:

Make a temporary directory where you will store the output log and the exit status of each program, something like:

tmpDir=$(mktemp -d ...)

Then start each program within a subshell, in the background, writing to a log file and saving the exit status, something like:

( prog1 > "$tmpDir/1.log" ; echo $? > "$tmpDir/1.es" ) &

Now go into a loop, waiting for all to finish. You can either test whether the exit status file exists, or use pgrep to know when complete.

When all are complete, exit the loop and cat the log for each process where the exit status is non-zero.

user_28366210
Related