How to safely reinterpret Vec<f64> as Vec<num_complex::Complex<f64>> with half the size?

user_2520171 0
Viewed 819

I have complex number data filled into a Vec<f64> by an external C library (prefer not to change) in the form [i_0_real, i_0_imag, i_1_real, i_1_imag, ...] and it appears that this Vec<f64> has the same memory layout as a Vec<num_complex::Complex<f64>> of half the length would be, given that num_complex::Complex<f64>'s data structure is memory-layout compatible with [f64; 2] as documented here. I'd like to use it as such without needing a re-allocation of a potentially large buffer.

I'm assuming that it's valid to use from_raw_parts() in std::vec::Vec to fake a new Vec that takes ownership of the old Vec's memory (by forgetting the old Vec) and use size / 2 and capacity / 2, but that requires unsafe code. Is there a "safe" way to do this kind of data re-interpretation?

The Vec is allocated in Rust as a Vec<f64> and is populated by a C function using .as_mut_ptr() that fills in the Vec<f64>.

My current compiling unsafe implementation:

extern crate num_complex;

pub fn convert_to_complex_unsafe(mut buffer: Vec<f64>) -> Vec<num_complex::Complex<f64>> {
    let new_vec = unsafe {
        Vec::from_raw_parts(
            buffer.as_mut_ptr() as *mut num_complex::Complex<f64>,
            buffer.len() / 2,
            buffer.capacity() / 2,
        )
    };
    std::mem::forget(buffer);
    return new_vec;
}

fn main() {
    println!(
        "Converted vector: {:?}",
        convert_to_complex_unsafe(vec![3.0, 4.0, 5.0, 6.0])
    );
}
rust unsafe
1 Answers

Is there a "safe" way to do this kind of data re-interpretation?

No. At the very least, this is because the information you need to know is not expressed in the Rust type system but is expressed via prose (a.k.a. the docs):

Complex<T> is memory layout compatible with an array [T; 2].

— Complex docs

If a Vec has allocated memory, then [...] its pointer points to len initialized, contiguous elements in order (what you would see if you coerced it to a slice),

— Vec docs

Arrays coerce to slices ([T])

— Array docs

Since a Complex is memory-compatible with an array, an array's data is memory-compatible with a slice, and a Vec's data is memory-compatible with a slice, this transformation should be safe, even though the compiler cannot tell this.

This information should be attached (via a comment) to your unsafe block.

I would make some small tweaks to your function:

  • Having two Vecs at the same time pointing to the same data makes me very nervous. This can be trivially avoided by introducing some variables and forgetting one before creating the other.

  • Remove the return keyword to be more idiomatic

  • Add some asserts that the starting length of the data is a multiple of two.

  • As rodrigo points out, the capacity could easily be an odd number. To attempt to avoid this, we call shrink_to_fit. This has the downside that the Vec may need to reallocate and copy the memory, depending on the implementation.

  • Expand the unsafe block to cover all of the related code that is required to ensure that the safety invariants are upheld.

pub fn convert_to_complex(mut buffer: Vec<f64>) -> Vec<num_complex::Complex<f64>> {
    // This is where I'd put the rationale for why this `unsafe` block
    // upholds the guarantees that I must ensure. Too bad I
    // copy-and-pasted from Stack Overflow without reading this comment!
    unsafe {
        buffer.shrink_to_fit();

        let ptr = buffer.as_mut_ptr() as *mut num_complex::Complex<f64>;
        let len = buffer.len();
        let cap = buffer.capacity();

        assert!(len % 2 == 0);
        assert!(cap % 2 == 0);

        std::mem::forget(buffer);

        Vec::from_raw_parts(ptr, len / 2, cap / 2)
    }
}

To avoid all the worrying about the capacity, you could just convert a slice into the Vec. This also doesn't have any extra memory allocation. It's simpler because we can "lose" any odd trailing values because the Vec still maintains them.

pub fn convert_to_complex(buffer: &[f64]) -> &[num_complex::Complex<f64>] {
    // This is where I'd put the rationale for why this `unsafe` block
    // upholds the guarantees that I must ensure. Too bad I
    // copy-and-pasted from Stack Overflow without reading this comment!
    unsafe {
        let ptr = buffer.as_ptr() as *mut num_complex::Complex<f64>;
        let len = buffer.len();

        std::slice::from_raw_parts(ptr, len / 2)
    }
}
user_1554230
Related