Not possible to get bash output to file

user_9463681 0
Viewed 95

I have Vowpal Wabbit daemon running, which I started with following command:

vw -i X_b123.model -t --quiet --daemon --port 26542 -r /dev/stdout

Then i try to send something to daemon:

echo 'somedataforwovpal' | netcat localhost 26542 -q1

I get answer, like this (both lines are the answer):

1:-0.0268077 2:-0.0990701 3:-0.154975
2

For now, everything is perfect and correct. What i want, is just send this output to file. And this is the point that drives me crazy. Why? Usually i would do it simple, by sending stdout to file like this:

echo 'somedataforwovpal' | netcat localhost 26542 -q1 > myfile.txt

Well, this worked only partially. The file was created, but it contains only the second line of output (just the number 2), and first line 1:-0.0268077 2:-0.0990701 3:-0.154975 is still print to console.

So my idea was, that the second part is going to stderr, so I tried several following ways to save stderr/stdout output:

echo 'somedataforwovpal' | netcat localhost 26542 -q1 2> myfile.txt
echo 'somedataforwovpal' | netcat localhost 26542 -q1 2>&1 >  myfile.txt
echo 'somedataforwovpal' | netcat localhost 26542 -q1 &>  myfile.txt
echo 'somedataforwovpal' | netcat localhost 26542 -q1 > myfile.txt 2>&1
echo 'somedataforwovpal' | netcat localhost 26542 -q1 >> myfile.txt
script -c "echo '1 |w auto_t dum_qt |f auto_t dum_qt |m qm_pos_2' | netcat localhost 26542 -q1" myfile.txt

Did not worked, still same. None of these methods, as you can see, i even tried script, but still same. This really drives my crazy, please, is there anybody who could save me?

vowpalwabbit stdout shell unix bash
3 Answers

It is possible to bypass normal redirections, but generally not a great idea. For example:

#! /bin/env bash
echo $* >> $(tty)

running it works.

$: bypass foo >log 2>&1
foo

Nothing goes in the log, either.

$: ls -l log
-rw-r--r-- 1 P2759474 1049089 0 Nov  7 12:16 log

I'm sorry, I don't have a good solution for you, but this may at least help you wrap your head around how this could be happening.

user_86565520

Why would appending every new line not work?

echo 'somedataforwovpal' | netcat localhost 26542 -q1 >> myfile.txt

From the bash man page:

Appending Redirected Output

Redirection of output in this fashion causes the file whose name results from the expansion of word to be opened for appending on file descriptor n, or the standard output (file descriptor 1) if n is not specified. If the file does not exist it is created.

The general format for appending output is:

[n]>>word

Does this give any clues?

echo 'somedataforwovpal' | netcat localhost 26542 -q1 | cat -net
user_75206560

If you want to save all output (stdout + stderr + explicit tty) to a file you may use the script utility:

$ script /tmp/session.log
Script started, file is /tmp/session.log

$ some_command args...
... output of some_command args ...

$ exit
Script done, file is /tmp/session.log

Now /tmp/session.log contains everything that was captured on screen during the session.

The man page for script warns:

script places everything in the log file, including linefeeds and backspaces. This is not what the naive user expects.

You may use a perl-script like the below to strip the final output from unwanted junk (not very scientific, but works for me)

#!/usr/bin/perl -w
#
# Filter to clean script (and other terminal/interactive) junk output
#
while (<>) {
    # ANSI terminal sequences (empirical)
    s/\033\[\??(?:\d*(?:;\d*)*)?[hlmHJKL]//g;
    s/\033[=>]//g;

    # Backspace/delete sequences, slow/safe way from the inside out
    s/[^\010]\010//g;

    # Other CTRL & non-wanted ASCII chars
    tr/\000-\010\013-\037\177//d;

    print;
}
user_12960440
Related