18

I have a script executing commands like:

export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH;./some_app -i $INDEX | tee $LOG
echo "Number of errors: $(grep "ERROR" $LOG | wc -l)"

The problem is probably in the pipe to tee. It does not seem to get the whole output. When the application quits last few lines of the output (usually those containing a fatal error) are missing. When I run the app without pipe to tee I get them in the output.

How can I force script to wait for tee to complete processing of all output?

Ladislav Mrnka
  • 283
  • 1
  • 2
  • 6

2 Answers2

30

The fatal error is probably coming out in STDERR (2), not STDOUT (1). You can redirect STDERR into STDOUT with 2>&1 and then the pipe should capture it too.

./some_app -i $INDEX 2>&1 | tee $LOG

If you have buffering issues on top, you could force it into an unbuffered state:

stdbuf -o0 ./some_app -i $INDEX 2>&1 | tee $LOG
Oli
  • 289,791
  • 117
  • 680
  • 835
  • Good, we are getting closer. Now I see that fatal error is being printed but again it is not complete. The line with the error just ends in the middle and echo output continues. There is still some problem with flushing buffer or simply waiting on that part to complete. – Ladislav Mrnka Jun 23 '15 at 10:33
  • 1
    Edited. Fairly rare in my experience that something so completely slips through buffers on exit but it's worth a go. – Oli Jun 23 '15 at 10:38
  • 1
    Done! Thank you. I may be asking too many question but does anyone understand why I need to turn off buffering when piping to other process? – Ladislav Mrnka Jun 23 '15 at 10:40
  • @Oli A very good one! – Pilot6 Jun 23 '15 at 10:50
7

As the error messages are normally shown on STDERR (File descriptor 2), you need to redirect both the STDOUT and STDERR to tee:

./some_app -i "$INDEX" |& tee "$LOG"

When you do ./some_app -i $INDEX | tee $LOG you are only redirecting the STDOUT to tee.

|& will cause both the STDOUT and STDERR to be redirected.

If you cant to redirect only the STDOUT (As you were):

./some_app -i "$INDEX" | tee "$LOG"

On the other hand if you want to redirect only the STDERR:

./some_app -i "$INDEX" 2>&1 >/dev/null | tee "$LOG"
heemayl
  • 90,425
  • 20
  • 200
  • 267