How to send a vector to spawned thread in Rust

user_13284735 0
Viewed 169
[UPDATE] image with error was replaced with pure code (according to comments)

I'm newbie in Rust and came from JS. So I have a problem with threads.

Question 1

I'd like to try to speed up my program, but I don't understand how to send my batch to function.

fn read_lines(reader: BufReader<File>) -> Vec<JoinHandle<()>> {
    let mut batch = vec![];
    let mut children_th = vec![];
    for line in reader.lines() {
        let line_str = line.expect("invalid line");
        if batch.len() >= BATCH_SIZE && !line_str.contains('[') {
            batch.push(line_str);
            children_th.push(thread::spawn(|| process_data(batch.clone())));
            batch.clear();
        }
    }
    children_th
}

fn process_data(lines: Vec<String>) {
    let mut rus_word = "".to_string();
    let mut definition: String = "".to_string();
    let mut parsed_result: Vec<VedyDictionary> = vec![];
    for line_str in lines {
        if line_str.trim().chars().count() > 1 {
            if !line_str.contains('[') {
                if !rus_word.is_empty() {
                    parsed_result.push(parse_to_dictionary(rus_word, &definition));
                    definition = "".to_string();
                }
                rus_word = line_str.trim().to_string();
            } else {
                definition.push_str(line_str.trim());
                definition.push('\n');
            }
        }
    }
    write_jsons(&parsed_result);
}

I'm getting this error in that case:

error[E0382]: borrow of moved value: `batch`
  --> src/main.rs:41:12
   |
36 |     let mut batch = vec![];
   |         --------- move occurs because `batch` has type `Vec<std::string::String>`, which does not implement the `Copy` trait
...
41 |         if batch.len() >= BATCH_SIZE && !line_str.contains('[') {
   |            ^^^^^^^^^^^ value borrowed here after move
...
44 |             children_th.push(thread::spawn(move || process_data(batch.clone(), index)));
   |                                            ------- value moved into closure here, in previous iteration of loop

I found a workaround here, but it seems ugly. I think I'm doing smth wrong. Could someone recommend something? Workaround:

fn read_lines(reader: BufReader<File>) -> Vec<JoinHandle<()>> {
    let mut batch = vec![];
    let mut children_th = vec![];
    for line in reader.lines() {
        let line_str = line.expect("invalid line");
        if batch.len() >= BATCH_SIZE && !line_str.contains('[') {
            batch.push(line_str);
            let batch_clone = batch.clone(); // clone here
            children_th.push(thread::spawn(move || process_data(batch_clone))); // move cloned variable here
            batch.clear();
        } else {
            batch.push(line_str);
        }
    }
    children_th
}

Question 2

Is it possible to do smth like that:

fn read_lines(reader: BufReader<File>) -> Vec<JoinHandle<()>> {
    let mut batch = vec![];
    let mut children_th = vec![];
    for line in reader.lines() {
        let line_str = line.expect("invalid line");
        if batch.len() >= BATCH_SIZE && !line_str.contains('[') {
            batch.push(line_str);
            children_th.push(thread::spawn(process_data(batch.clone()))); // no || clouser
            batch.clear();
        }
    }
    children_th
}

Thanks!

rust thread-safety multithreading
1 Answers

I notice that you immediately clear the local copy of the Vec after spawning the new thread. In other words, you want to give the contents of the Vec to a new thread and start over with an empty Vec.

When you move the value in a variable in Rust, the variable effectively becomes uninitialized again, meaning you can assign a new value to it even after moving it. So, you don't need to clone at all. Move the local Vec into the new thread closure, and assign a new Vec after doing so:

batch.push(line_str);
children_th.push(thread::spawn(move || process_data(batch)));
// The batch variable is now "dead" but we can revive it by assigning
// a new value to it:
batch = vec![];

This avoids a needless clone of the entire Vec's contents.

user_5012500
Related