How to redirect and append both standard output and standard error to a file with Bash

user_63051 0
Viewed 900865

To redirect standard output to a truncated file in Bash, I know to use:

cmd > file.txt

To redirect standard output in Bash, appending to a file, I know to use:

cmd >> file.txt

To redirect both standard output and standard error to a truncated file, I know to use:

cmd &> file.txt

How do I redirect both standard output and standard error appending to a file? cmd &>> file.txt did not work for me.

io-redirection stderr append stdout bash
9 Answers
cmd >>file.txt 2>&1

Bash executes the redirects from left to right as follows:

  1. >>file.txt: Open file.txt in append mode and redirect stdout there.
  2. 2>&1: Redirect stderr to "where stdout is currently going". In this case, that is a file opened in append mode. In other words, the &1 reuses the file descriptor which stdout currently uses.
user_958100

There are two ways to do this, depending on your Bash version.

The classic and portable (Bash pre-4) way is:

cmd >> outfile 2>&1

A nonportable way, starting with Bash 4 is

cmd &>> outfile

(analog to &> outfile)

For good coding style, you should

  • decide if portability is a concern (then use the classic way)
  • decide if portability even to Bash pre-4 is a concern (then use the classic way)
  • no matter which syntax you use, don't change it within the same script (confusion!)

If your script already starts with #!/bin/sh (no matter if intended or not), then the Bash 4 solution, and in general any Bash-specific code, is not the way to go.

Also remember that Bash 4 &>> is just shorter syntax — it does not introduce any new functionality or anything like that.

The syntax is (beside other redirection syntax) described in the Bash hackers wiki.

user_1041620

In Bash 4 (as well as Z shell (zsh) 4.3.11):

cmd &>> outfile

just out of box.

user_1673620

Another approach:

If using older versions of Bash where &>> isn't available, you also can do:

(cmd 2>&1) >> file.txt

This spawns a subshell, so it's less efficient than the traditional approach of cmd >> file.txt 2>&1, and it consequently won't work for commands that need to modify the current shell (e.g. cd, pushd), but this approach feels more natural and understandable to me:

  1. Redirect standard error to standard output.
  2. Redirect the new standard output by appending to a file.

Also, the parentheses remove any ambiguity of order, especially if you want to pipe standard output and standard error to another command instead.

To avoid starting a subshell, you instead could use curly braces instead of parentheses to create a group command:

{ cmd 2>&1; } >> file.txt

(Note that a semicolon (or newline) is required to terminate the group command.)

user_1797150

Redirections from script himself

You could plan redirections from the script itself:

#!/bin/bash

exec 1>>logfile.txt
exec 2>&1

/bin/ls -ld /tmp /tnt

Running this will create/append logfile.txt, containing:

/bin/ls: cannot access '/tnt': No such file or directory
drwxrwxrwt 2 root root 4096 Apr  5 11:20 /tmp

Log to many different files

You could create two different logfiles, appending to one overall log and recreating another last log:

#!/bin/bash

if [ -e last.log ] ;then
    mv -f last.log last.old
fi
exec 1> >(tee -a overall.log /dev/tty >last.log)
exec 2>&1

ls -ld /tnt /tmp

Running this script will

  • if last.log already exist, rename them to last.old (overwriting last.old if they exist).
  • create a new last.log.
  • append everything to overall.log
  • output everything to the terminal.

Simple and combined logs

#!/bin/bash

[ -e last.err ] && mv -f last.err lasterr.old
[ -e last.log ] && mv -f last.log lastlog.old

exec 2> >(tee -a overall.err combined.log /dev/tty >last.err)
exec 1> >(tee -a overall.log combined.log /dev/tty >last.log)

ls -ld /tnt /tmp

So you have

  • last.log last run log file
  • last.err last run error file
  • lastlog.old previous run log file
  • lasterr.old previous run error file
  • overall.log appended overall log file
  • overall.err appended overall error file
  • combined.log appended overall error and log combined file.
  • still output to the terminal

And for interactive session, use stdbuf:

If you plan to use this in interactive shell, you must tell tee to not buffering his input/output:

# Source this to multi-log your session
[ -e last.err ] && mv -f last.err lasterr.old
[ -e last.log ] && mv -f last.log lastlog.old
exec 2> >(exec stdbuf -i0 -o0 tee -a overall.err combined.log /dev/tty >last.err)
exec 1> >(exec stdbuf -i0 -o0 tee -a overall.log combined.log /dev/tty >last.log)

Once sourced this, you could try:

ls -ld /tnt /tmp

Some explanations about exec and stdbuf commands:

  • Running forks by using $(...) or <(...) is done by running subshell wich will execute binaries in another subshell (subsubshell). The exec command tell shell that there are not further command in script to be run, so binary (stdbuf ... tee) will be executed as replacement process, at same level (no need to reserve more memory for running another sub-process).

    From bash's man page (man -P'less +/^\ *exec\ ' bash):

        exec [-cl] [-a name] [command [arguments]]
           If  command  is  specified,  it  replaces the
           shell.  No new process is created....

    This is not really needed, but reduce system footprint.

  • From stdbuf's man page:

    NAME
       stdbuf  -  Run COMMAND, with modified buffering
       operations for its standard streams.

    This will tell system to use unbuffered I/O for tee command. So all outputs will be updated immediately, when some input are coming.

user_17656580

If you care about the ordering of the content of the two streams, see @ed-morton 's answer to a similar question, here.

user_83412740
Related