Capture the output of subprocess.run() but also print it in real time?

user_3064538 0
Viewed 3939

I would like to run a command using subprocess.run() and then get its stdout/stderr as a string, but I want the subprocess to also print its output to the console normally while running. If I do

result = subprocess.run(['ls', '-al'])

then I can see the output printed to my console but I can't access the output after the command runs. If I do

result = subprocess.run(['ls', '-al'], capture_output=True, text=True)

I can access result.stdout and result.stderr but nothing is printed to the console while the command is running. Can I have both printing to the console and saving to result.stdout?

subprocess python
1 Answers

From the documentation of subprocess.run :

Run the command described by args. Wait for command to complete, then return a CompletedProcess instance.
[...]
If capture_output is true, stdout and stderr will be captured. When used, the internal Popen object is automatically created with stdout=PIPE and stderr=PIPE.

The docs for subprocess.PIPE say :

Special value that can be used as the stdin, stdout or stderr argument to Popen and indicates that a pipe to the standard stream should be opened. Most useful with Popen.communicate().

The doc for the Popen constructor parameter stdout :

stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are PIPE, DEVNULL, an existing file descriptor (a positive integer), an existing file object, and None.

So using capture_output=True is a no-go, because the output will be stored in a pipe for you to read after the call finishes.

The simpler is for you to use subprocess.Popen as @MatBBastos suggested, with wich you can communicate (repeatedly sending content to stdin and receiving content from stdout/stderr). The solution linked is a bit dated (cf its own comments; Python 2) but should work well. A related solution is this one.

To keep using subprocess.run, you will have to provide a file descriptor as stdout parameter, which I don't know how would have to redirect to a file object that does what you want : writing to the standard stream, but also keeping a copy in memory for later use.
There are docs in the io module, and a lot of questions on Stack Overflow about doing things like that, but it is notably more difficult than the other way.

user_113841840
Related