Is an array required to be initialized if I will read file contents into it?

Admin 1
Viewed 984

I am initializing buf unnecessarily with all zeros before immediately writing over its contents with read_exact:

fn parse<R: Read>(r: &mut R) -> Result<()> {
    let mut buf = [0u8; 1024];
    r.read_exact(&mut buf)?;
    // Do things to buf
    Ok(true)
}

While I don't think the 0-initialization is very time-consuming, it doesn't seem necessary. Is there a way around this?

rust arrays
2 Answers

Yes, such an initialization is required in safe Rust. The compiler isn't necessarily able to tell that the combination of read_exact and ? will prevent accesses to any uninitialized data from the array. The optimizing passes may make it better, but that cannot be counted on to work.

You could instead read into a Vec, which internally guarantees to never allow access to the uninitialized memory:

fn parse<R: Read>(r: &mut R) -> io::Result<()> {
    let mut buf = vec![]; // Can optionally use `Vec::with_capacity`
    r.take(1024).read_to_end(&mut buf)?;
    // Do things to buf
    Ok(())
}

In unsafe Rust, you can use mem::uninitialized. Make sure you understand all the terrible little nuances involved before choosing such a situation! Reading uninitialized memory is undefined behavior in Rust, so you must make absolutely sure that you prevent it from ever happening.

Here, we rely on:

  1. The fact that read_exact will return an error if it didn't populate all of the bytes.
  2. The fact that the destructor of [u8; 1024] will not read the bytes. This is important path to consider when an error occurs or if a panic is raised.
fn parse<R: Read>(r: &mut R) -> io::Result<()> {
    unsafe {
        let mut buf: [u8; 1024] = mem::uninitialized();
        r.read_exact(&mut buf)?;
        // Do things to buf
    }
    Ok(())
}

I wouldn't think this is would be worth it without heavy profiling.

user_1554230

There is currently no way to do this with read_exact in Rust outside of unsafe blocks. However, you can probably get what you want in this particular case by using std::io::BufReader::with_capacity instead of allocating your own mut [u8].

If you don't care about being able to set the buffer length yourself, you could have your function require the std::io::BufRead trait for its argument instead of Read.

Either of the approaches I mentioned will allow you to use fill_buf and consume instead of read_exact, which should give you the performance you want.

user_22725060
Related