Bash tail command and multiple pipes

user_4810787 0
Viewed 2064

I am currently facing a problem with the command

while sleep 0.3; do echo "1"; done | tail -n +3 | grep --line-buffered "1"

I wanted an output that looks like:

[nothing during 0.6s]
1
1
...

but had:

[nothing forever]

I found this question and read this blog post but as the --line-buffered option is set for grep, I think this is not a buffering problem. My researches did not led me to other solutions, I hope I did not miss something obvious.

I am using zsh 5.7.1 (x86_64-pc-linux-gnu) if it is important.

pipe bash
2 Answers

As Barmar mentioned in the comments, this is getting swallowed up in a 4KB stdio buffer. You can use unbuffer -p to avoid it:

while sleep 0.3; do echo "1"; done | unbuffer -p tail -n +3 | grep 1

Or you could also use stdbuf in line-buffering mode:

while sleep 0.3; do echo "1"; done | stdbuf -oL tail -n +3 | grep 1
user_14268910

Depending on the actual situation, you might be able to avoid the problem (caused by the pipe from tail to grep) by doing everything with a single more powerful tool, like awk:

while sleep 0.3; do echo "1"; done | awk 'NR > 2 && /1/'

But of course, if you need to pipe the output from awk into something else, you just have the same problem all over again. But if your version of awk supports it, you can explicitly flush output:

while sleep 0.3; do echo "1"; done | awk 'NR > 2 && /1/ {print $0; fflush()}' | | while read x; do echo "read '$x'"; done

...or presumably use unbuffer -p or stdbuf -oL on awk (see @JeffBowman's answer), if you have those (I don't, so I can't test them).

user_898170
Related