How to parallelize a UNIX for-loop?

user_19565734 0
Viewed 35

I apologize for the somewhat simple question (I am new to UNIX programming). I am using an HPC system and would like to parallelize a task in a for loop (in this case, a simple unzip to be applied to several files). How could I do this? I thought that by requesting multiple cores the parallelization would start automatically, but actually the program is operating sequentially. Thank you very much!

for i in *.zip; do unzip "$i" -d "${i%%.zip}"; done
slurm parallel-processing for-loop unix bash
1 Answers

In bash it would look something like:

for item in bunch-of-items; do
  (
    the loop body
    is here
  ) &
done

Where the parentheses group the commands, and the whole loop body is put in the background.

If the rest of your program needs all the background jobs to complete, use the wait command.

user_75520
Related