Writing to stdio & reading from stdout in Rust Command process

Question

I'll try to simplify as much as possible what I'm trying to do accomplish but in a nutshell here is my problem:

I am trying to spawn the node shell as a process in Rust. I would like to pass to the process' stdin javascript code and read the nodejs output from stdout of the process. This would be an interactive usage where the node shell is spawned and keeps receiving JS instructions and executing them.

I do not wish to launch the nodejs app using a file argument.

I have read quite a bit about std::process::Command, tokio and why we can't write and read to a piped input using standard library. One of the solutions that I kept seeing online (in order to not block the main thread while reading/writing) is to use a thread for reading the output. Most solutions did not involve a continuous write/read flow.

What I have done is to spawn 2 threads, one that keeps writing to stdin and one that keeps reading from stdout. That way, I thought, I won't be blocking the main thread. However my issue is that only 1 thread can actively be used. When I have a thread for stdin, stdout does not even receive data.

Here is the code, comments should provide more details

pub struct Runner {
    handle: Child,
    pub input: Arc>,
    pub output: Arc>,
    input_thread: JoinHandle<()>,
    output_thread: JoinHandle<()>,
}

impl Runner {
    pub fn new() -> Runner {
        let mut handle = Command::new("node")
            .stdin(Stdio::piped())
            .stdout(Stdio::piped())
            .spawn()
            .expect("Failed to spawn node process!");


        // begin stdout thread part
        let mut stdout = handle.stdout.take().unwrap();
        let output = Arc::new(Mutex::new(String::new()));
        let out_clone = Arc::clone(&output);
        let output_thread = spawn(move || loop {
            // code here never executes...why ?
            let mut buf: [u8; 1] = [0];
            let mut output = out_clone.lock().unwrap();

            let what_i_read = stdout.read(&mut buf);
            println!("reading: {:?}", what_i_read);
            match what_i_read {
                Err(err) => {
                    println!("{}] Error reading from stream: {}", line!(), err);
                    break;
                }
                Ok(bytes_read) => {
                    if bytes_read != 0 {
                        let char = String::from_utf8(buf.to_vec()).unwrap();
                        output.push_str(char.as_str());
                    } else if output.len() != 0 {
                        println!("result: {}", output);
                        out_clone.lock().unwrap().clear();
                    }
                }
            }
        });

        // begin stdin thread block
        let mut stdin = handle.stdin.take().unwrap();
        let input = Arc::new(Mutex::new(String::new()));
        let input_clone = Arc::clone(&input);

        let input_thread = spawn(move || loop {
            let mut in_text = input_clone.lock().unwrap();

            if in_text.len() != 0 {
                println!("writing: {}", in_text);
                stdin.write_all(in_text.as_bytes()).expect("!write");
                stdin.write_all("
".as_bytes()).expect("!write");
                in_text.clear();
            }
        });

        Runner {
            handle,
            input,
            output,
            input_thread,
            output_thread,
        }
    }
    // this function should receive commands
    pub fn execute(&mut self, str: &str) {
        let input = Arc::clone(&self.input);
        let mut input = input.lock().unwrap();

        input.push_str(str);
    }
}

In the main thread I'd like use this as

let mut runner = Runner::new();
runner.execute("console.log('foo'");
println!("{:?}", runner.output);

I am still new to Rust but at least I passed the point where the borrow checker makes me bang my head against the wall, I am starting to find it more pleasing now :)

user_11998029 · Answer

Do you set up a loop in the thread to wait for input? how to wait for the end of the program.

You now, if you do

use std::{thread};

fn main() {
    let _output_thread = thread::spawn(move || {
            println!("done");
    });
}

The end of the program is sometimes more quick of thread and you not see the output "done".

If you do a loop like this, is the same

use std::{thread};

fn main() {
    //let output_thread = spawn(move || loop {
            // code here never executes...why ?
    let _output_thread = thread::spawn(move || {
        //-------------------------------------^---
        loop {
            println!("done");
        }
    });
}

If you wait it work well.

use std::{thread, time};

fn main() {
    //let output_thread = spawn(move || loop {
            // code here never executes...why ?
    let _output_thread = thread::spawn(move || {
        //-------------------------------------^---
        loop {
            println!("done");
        }
    });

    thread::sleep(time::Duration::from_secs(5));
}

A good way to work with 2 tread is like expalned in https://doc.rust-lang.org/std/thread/fn.spawn.html, withe channel and join

use std::thread;
use std::sync::mpsc::channel;

let (tx, rx) = channel();

let sender = thread::spawn(move || {
    tx.send("Hello, thread".to_owned())
        .expect("Unable to send on channel");
});

let receiver = thread::spawn(move || {
    let value = rx.recv().expect("Unable to receive from channel");
    println!("{}", value);
});

sender.join().expect("The sender thread has panicked");
receiver.join().expect("The receiver thread has panicked");

Writing to stdio & reading from stdout in Rust Command process

1 Answers