POSIX sh: how do compound commands work with pipes?

user_13168258 0
Viewed 112

I'm trying to understand how groups of sequential commands (named "lists" in GNU bash manual) work with pipes in POSIX sh definition or its implementations. For example, the following code:

if test-expr1; then
  cmd1; cmd2; cmd3; ...
fi | { cmd4; cmd5; ...; } | cmd6

Here, the pipeline has two sequential "lists" and a single command, where cmd2 executes after cmd1 finishes and cmd5 after cmd4. However, all three are run in parallel so the pipeline doesn't block.

My first guess was that for each of these "lists", a subshell is created, with a separate PID, which executes all commands in sequence, but doesn't block the shell from spawning other processes — which would happen if shell started waiting for cmd1 to complete. This was incorrect, though, as echo $$ still reports the same PID even in piped command lists.

The question is: how exactly are fork()/waitpid() used to achieve the correct behavior?

sh posix shell bash
2 Answers
if test-expr1; then
  cmd1; cmd2; cmd3; ...
fi | { cmd4; cmd5; ...; } | cmd6

would be rendered in a parse tree as:

                                 _______________|
                                /                \
               ________________|___               \
              /                    \               \
          IF__________________      \               \
         /  \                 \      \               \
test-expr1   cmd1; cmd2; cmd3  (nil)  cmd4; cmd5;     cmd6

Looking at the leaves:

  1. test-expr1: might fork, if it involves running a command, otherwise needn't.
  2. cmd1; ... : likely fork, unless they are internals. One forked shell can run these sequentially
  3. cmd4; cmd5: likely fork, unless internals. One forked shell can run these seq.
  4. cmd6: likely fork, unless internal. One forked shell can exec this.

The IF statemnet (1, 2) is piped into (3). (3) is piped into 4. so, if your command was:

if /bin/echo You; then /bin/echo say; /bin/echo hi; fi | { cat -; echo I; echo say; echo Lo;  } | cat -;

you get the output:

You
say
hi
I
say
Lo
user_46644150

This was incorrect, though, as echo $$ still reports the same PID even in piped command lists.

This was correct, and your method of checking that was incorrect. From bash manual:

($$) Expands to the process ID of the shell. In a () subshell, it expands to the process ID of the invoking shell, not the subshell.

So you checked that the invoking shell is the same in all subshells. It is.

user_90727530
Related