How do I detect a failed subprocess in a bash read statement?

Question

In bash we can set an environment variable from a sequence of commands using read and a pipe to a subprocess. But I'm having trouble detecting errors in my processing in one edge case - a part of the subprocess pipeline producing some output before erroring.

A simplified example which takes an input file, looks for a line starting with "foo" and sets var to the first word on that line is:

set -e
set -o pipefail
set -o nounset

die() {
  echo $1 > /dev/stderr
  exit 1
}

read -r var rest < <( \
    cat data.txt \
    | grep foo \
    || die "PIPELINE" \
) || die "OUTER"

echo "var=$var"

Running this with data.txt like

blah
zap foo awesome
bang foo

will output

var=zap

Running this on a data.txt file that doesn't contain foo outputs (to stderr)

DEAD: PIPELINE
DEAD: OUTER

This is all as expected.

We can introduce another no-op stage like cat at the end of the process

...
read -r var rest < <( \
    cat data.txt \
    | grep foo \
    | cat \
    || die "PIPELINE" \
) || die "OUTER"
...

and everything continues to work.

But if the additional stage is paste -s -d' ' and the input does not contain "foo" the output is

var=
DEAD: PIPELINE

Which seems to show that the pipeline errors, but read succeeds with an empty line. (It looks like paste -s -d' ' outputs a line of output even when its input is empty.)

Is there a simple way to detect this failure of the pipeline, and cause the main script to error out?

I guess I could check that the variable is not empty - but this is a simplified version - I'm actually using sed and paste to join multiple lines to set multiple variables, like

read -r v1 v2 v3 rest < <( \
   cat data.txt \
   | grep "^foo=" \
   | sed -e 's/foo=//' \
   | paste -s -d' ' \
   || die "PIPELINE"
) || die "OUTER"

user_2072269 · Answer

You could use another grep to see if the output of paste contained something:

read -r var rest < <( \
    cat data.txt \
    | grep foo \
    | paste -s -d' '  \
    | grep . \
    || die "PIPELINE" \
) || die "OUTER"

user_221955 · Answer

In the end I went with two different solutions depending on the context.

The first was to pipe the results to a temporary file. This will process the entire file before performing the read, and thus any failures in the pipe will cause the script to fail.

cat data.txt \
    | grep "^foo=" \
    | sed -e 's/foo=//' \
    | paste -s -d' ' \
    > $TMP/result.txt
    || die "PIPELINE"

read -r var rest < $TMP/result.txt || die "OUTER"

The second was to just test that the variables were set. While this meant there was a bunch of duplication that I wanted to avoid, it seemed the most bullet-proof solution.

read -r var rest < <( cat data.txt \
    | grep "^foo=" \
    | sed -e 's/foo=//' \
    | paste -s -d' ' \
    || die "PIPELINE"
    ) || die "OUTER"

 [ ! -z "$var" ] || die "VARIABLE NOT SET"

How do I detect a failed subprocess in a bash read statement?

2 Answers