Borrow checker and shared I/O

user_237815 0
Viewed 87

I'm hitting a problem in my code where multiple structs need to send data to a shared output sink and the borrow checker doesn't like it.

struct SharedWriter {
    count: u32,
}

impl SharedWriter {
    pub fn write(&mut self) {
        self.count += 1;
    }
}

struct Container<'a> {
    writer: &'a mut SharedWriter,
}

impl Container<'_> {
    pub fn write(&mut self) {
        self.writer.write();
    }
}

pub fn test() {
    let mut writer = SharedWriter { count: 0 };

    let mut c0 = Container {
        writer: &mut writer,
    };

    let mut c1 = Container {
        // compiler chokes here with:
        // cannot borrow `writer` as mutable more than once at a time
        writer: &mut writer,
    };

    c0.write();
    c1.write();
}

I understand the problem and why it's happening; you can't borrow something as mutable more than once at a time.

What I don't understand is a good general solution. This pattern happens a lot. You've got a common output sink, like a file or a socket or a database, and you want to feed multiple streams of data to it. It has to be mutable if it maintains any kind of state. It has to be just a single entity if it holds any resources.

You could pass a reference to the sink in every single write() method (write(&mut writer, some_data)), but this clutters the code and will get called (in my particular app) millions of times per second. I'm speculating that there is some extra overhead in passing this parameter over and over.

Is there some syntax that will get past this problem?

borrow-checker rust
2 Answers

Interior mutability.

In your case the easiest way is probably to use RefCell. It will have some runtime overhead, but it is safe.

use std::cell::RefCell;

struct SharedWriter {
    count: RefCell<u32>,
}

impl SharedWriter {
    pub fn new(count: u32) -> Self {
        Self { count: RefCell::new(count) }
    }
    
    pub fn write(&self) {
        *self.count.borrow_mut() += 1;
    }
}

If the data is Copy (like u32, in case this is your real data), you may want to use Cell. It is applicable to less types but zero-cost:

use std::cell::Cell;

struct SharedWriter {
    count: Cell<u32>,
}

impl SharedWriter {
    pub fn new(count: u32) -> Self {
        Self { count: Cell::new(count) }
    }
    
    pub fn write(&self) {
        self.count.set(self.count.get() + 1);
    }
}

There are more interior mutability primitives (for example, UnsafeCell for zero-cost but unsafe access, or mutexes and atomics for thread safe mutation).

user_78843050

One option would be to use a channel. Here is an example of how that might look. This also has the added benefit of allowing you to scale across multiple threads with your io. It takes a handler which it runs in a loop on a new thread. It blocks until a value sent through the sender is received then calls func with a given value and a mutable reference to the handler. The thread exits when all the senders have been dropped. However one downside of this approach is the channel only works in one direction.

use std::sync::mpsc::{channel, Sender};
use std::thread::{self, JoinHandle};

pub fn create_shared_io<T, H, F>(mut handler: H, mut func: F) -> (JoinHandle<H>, Sender<T>)
where
    T: 'static + Send,
    H: 'static + Send,
    F: 'static + FnMut(&mut H, T) + Send,
{
    let (send, recv) = channel();

    let join_handle = thread::spawn(move || loop {
        let value = match recv.recv() {
            Ok(v) => v,
            Err(_) => break handler,
        };

        func(&mut handler, value);
    });

    (join_handle, send)
}

And then it can be used similarly to your example. Since no data was passed in your example, it sends () as a placeholder.

pub fn main() {
    let writer = SharedWriter { count: 0 };
    println!("Starting!");
    
    let (join_handle, sender) = create_shared_io(writer, |writer, _| {
        writer.count += 1;
        println!("Current count: {}", writer.count);
    });

    let mut c0 = Container {
        writer: sender.clone(),
    };

    let mut c1 = Container {
        writer: sender,
    };

    c0.write();
    c1.write();
    
    // Ensure the senders are dropped before we join the io thread to avoid possible deadlock
    // where the compiler attempts to drop these values after the join.
    std::mem::drop((c0, c1));
    
    // Writer is returned when the thread is joined
    let writer = join_handle.join().unwrap();
    println!("Finished!");
}

struct SharedWriter {
    count: u32,
}

struct Container {
    writer: Sender<()>,
}

impl Container {
    pub fn write(&mut self) {
        self.writer.send(()).unwrap();
    }
}
user_59876690
Related