Run command, stream stdout/stderr and capture results

user_147390 0
Viewed 105

I'm trying to use std::process::Command to run a command and stream its stdout and stderr while also capturing a copy of stdout/stderr. I found I can use spawn.

This code will capture the output, but won't stream it to stdout/stderr while it's happening:

let mut child = command
    .envs(env)
    .stdout(Stdio::piped()) // <=== Difference here
    .spawn()
    .unwrap();

let output = child
    .wait_with_output().unwrap();

println!("Done {}", std::str::from_utf8(&output.stdout).unwrap());

This code will stream the output but not capture it:

let mut child = command
    .envs(env)
    .spawn()
    .unwrap();

let output = child
    .wait_with_output().unwrap();

println!("Done {}", std::str::from_utf8(&output.stdout).unwrap());

Is there a way to capture a command's output while also streaming it to the parent stdout/stderr?

rust
2 Answers

This is similar to how I stream the compilation and execution output on Rust Explorer.

To stream the output you can pipe the stdout and read it line by line using BufReader.

Playground

use std::io::BufRead;
use std::io::BufReader;
use std::process::Command;
use std::process::Stdio;

fn main() {
    // Compile code.
    let mut child = Command::new("bash")
        .args([
            "-c",
            "echo 'Hello'; sleep 3s; echo 'World'"
        ])
        .stdout(Stdio::piped())
        .spawn()
        .unwrap();
    let stdout = child.stdout.take().unwrap();

    // Stream output.
    let lines = BufReader::new(stdout).lines();
    for line in lines {
        println!("{}", line.unwrap());
    }
}
user_32982920

There might be a less verbose way to do this, but this is the solution I came up with.

Spawn the process with a piped io for stdout and stderr. Spawn a thread for stdout and stderr. In each thread read from the pipe and output directly to stdout or stderr then write the contents to a channel.

In the main thread wait for the process to finish, then join the threads and finally read each channel to get the contents of stdout and stderr.

let mut child = command
    .stdout(Stdio::piped())
    .stderr(Stdio::piped())
    .spawn()
    .unwrap();

let child_stdout = child
    .stdout
    .take()
    .expect("Internal error, could not take stdout");
let child_stderr = child
    .stderr
    .take()
    .expect("Internal error, could not take stderr");

let (stdout_tx, stdout_rx) = std::sync::mpsc::channel();
let (stderr_tx, stderr_rx) = std::sync::mpsc::channel();

let stdout_thread = thread::spawn(move || {
    let stdout_lines = BufReader::new(child_stdout).lines();
    for line in stdout_lines {
        let line = line.unwrap();
        println!("{}", line);
        stdout_tx.send(line).unwrap();
    }
});

let stderr_thread = thread::spawn(move || {
    let stderr_lines = BufReader::new(child_stderr).lines();
    for line in stderr_lines {
        let line = line.unwrap();
        eprintln!("{}", line);
        stderr_tx.send(line).unwrap();
    }
});

let status = child
    .wait()
    .expect("Internal error, failed to wait on child");

stdout_thread.join().unwrap();
stderr_thread.join().unwrap();

let stdout = stdout_rx.into_iter().collect::<Vec<String>>().join("");
let stderr = stderr_rx.into_iter().collect::<Vec<String>>().join("");

The channel isn't strictly needed. I originally wanted to mutate a string, but I'm new in Rust with threads and couldn't find any examples showing how to mutate a string in a thread and then read it back into main.

I'm accepting the other solution as it really answered my main question. I just wanted to post back to give everyone a fully-featured answer that does exactly what I originally asked

user_1473900
Related