shell scripting: error handling in piped command

Trifo 0
Viewed 20

I have a set of commands chained with pipes like this:

cat very_big_data.csv | awk -f ../bin/repair_preproc.awk |  tr -d "\n" | tr "\007" "\n" | sed 's/>  *</></g' > output.csv

The CSV file to be processen is quite big (10-20GB), so I would like to keep it in one stream.

I yould need to be able to catch errors potentially raising in the chained commands, for example I would need to be sure if the awk, the tr and the sed commands finished ALL successfully.

ksh
1 Answers

Whoops, finally I've found some answer.

In bash, there is az array, called PIPESTATUS, so you can check the exit status of every pipe member.

But I need to do it in AIX, I only have ksh. In ksh there is a shell option: pipefail. You can use this to get the first nonzero exit status from a pipe stricture. Well, in default AIX ksh it isn't working either, so you have to use ksh93 shell.

There it is.

Trifo0
Related