redirect from output file so can be captured

user_1023645 0
Viewed 71

A program foo writes to an output file and prints diagnostic information to stdout.

Thus:

foo -o ./out  -i ./input  > log

results in the valuable stuff in ./out and some mumbo jumbo in log.

I need to read ./out into another program (an R script I am writing).

The intermediate step of writing to file ./out and reading again is slow, and I need to do this for big ./out files, hundreds of times.

I would like to perform some kind of redirection so that foo writes to a file descriptor rather than a file ./out, which I can read into my script directly. But stdout is already used.

Here is what I tried:

in R:

library(data.table)
fread(cmd = "foo -o /dev/stdout -i ./input > /dev/null")

but I get an error:

  File '/data1/tmp/RtmpkKIpBm/fileeb789130da8e3' has size 0. Returning a NULL data.table.
io-redirection r bash
2 Answers
foo -o >(cat)  -i ./input  > log

Demo:

$ cat foo 
#!/bin/bash

if [ "$1" != "-o" ]; then
    exit 2
fi
echo "mumbo jumbo stdout" >&1
echo "valueable info" > "$2"
echo "mumbo jumbo stderr" >&2 # threw in stderr for good measure
$ ./foo -o x
mumbo jumbo stdout
mumbo jumbo stderr
$ cat x
valueable info
$ rm x
$ ./foo -o /dev/stdout
mumbo jumbo stdout
valueable info
mumbo jumbo stderr
$ ./foo -o /dev/stdout &>/dev/null
$ ./foo -o >(cat) &>/dev/null
valueable info

Explanation:

Every process has its own stdout. with ./foo -o /dev/stdout &>/dev/null, you're telling foo to output its valuable info into its own stdout, which is /dev/null. But with ./foo -o >(cat) &>/dev/null, you're telling foo to output its valuable info into some pipe, and that pipe goes to cat, whose stdout is not /dev/null, but rather inherited from the shell.

In the demo, the shell's stdout is the terminal, but if it was coming from R's fread(), both the shell's stdout and cat's stdout would go to where fread() can read them.

user_106789550

Try

foo -o /dev/fd/3 -i ./input 3>&1 1>/dev/null
  • Redirections are processed before commands are run
  • Redirections are processed left-to-right
  • 3>&1 causes anything written to file descriptor 3 to go to the current stdout
  • 1>/dev/null causes anything written to stdout to go to /dev/null
  • Writing to /dev/fd/3 then writes to the original stdout
  • The code should work with any POSIX-compliant shell on any system that supports the /dev/fd directory.
  • The /dev/fd directory is not part of POSIX, but it is supported on many Unix-like systems, including Linux, FreeBSD, macOS, and Solaris.
user_41543750
Related