piping long stdout into tee command sometimes will result in truncation

user_3672349 0
Viewed 538

At few examples I can see that tee doesn't wait for stdout to finish, and this results in some buggy behavior.

first example:
seq 50000|tee >(wc) >(head -2) >/dev/null
will output:

                1
                2
  12773   12774   65536

instead of

                1
                2
  50000   50000  288894

second example:
cat f.txt|grep abc|tee f.txt >/dev/null
at this example, the file update will work, only if the file is a short one, but if the file is a long one, sometimes tee will starts writing before cat finishes, and the file update will work only on the first 'cat' transition.

you can say that tee is only for redirecting to files, but I see also those usages with tee.
I know that pee command doesn't have this bug, but anyone knows a workaround to make tee to wait?
thanks :-)

tee shell linux bash
1 Answers

The problem you are experiencing can be reduced to this:

seq 50000 | tee >(wc) >(exit) >/dev/null

If one of the streams tee writes to returns an error, tee terminates. Because pipes are usually buffered with 4K buffer, it is enough for wc to grab some lines and process them.

head -n2 closes the stream after printing two lines.

You can see the error by telling tee to print something on errors:

# seq 50000 | tee --output-error=exit >(wc) >(exit) >/dev/null
tee: /dev/fd/62: broken pipe

You can remove the problem with -p or --output-error=warn to tell tee to continue after errors when writing to pipes:

seq 50000 | tee -p >(wc) >(head -n2) >/dev/null

or use a command that doesn't exit after processing the input, like sed:

seq 50000 | tee >(wc) >(sed -n '1,2p') >/dev/null
user_90727530
Related