Is the data in Vec<T> always densely packed?

user_353820 0
Viewed 270

It is a common pattern to see this 'shortcut' code in rust:

unsafe fn any_as_u8_slice<T: Sized>(p: &T) -> &[u8] {
    ::std::slice::from_raw_parts(
        (p as *const T) as *const u8,
        ::std::mem::size_of::<T>(),
    )
}

ie. Given a struct, unsafely convert the underlying pointer to &[u8] to read the bytes.

However, is it valid to take the same approach when using Vec<T>?

For example, this appears to work:

use std::mem::size_of;
use std::slice::from_raw_parts;

#[repr(C)]
#[derive(Debug, Copy, Clone)]
pub struct Point {
    pub x: u8,
    pub y: u8,
    pub z: u8,
}


fn as_bytes(data: &[Point]) -> &[u8] {
    unsafe { 
        let raw_pointer = data.as_ptr();
        from_raw_parts(raw_pointer as *const u8, size_of::<Point>() * data.len())
    }
}

fn main() {
    let points = vec![Point{x: 0u8, y: 1u8, z: 2u8}, Point{x: 3u8, y: 4u8, z: 5u8}];
    let slice = points.as_slice();
    println!("{:?}", slice);

    let bytes = as_bytes(slice);
    println!("{:?}", bytes);
    
    assert!(bytes.len() == 6);
    assert!(bytes[0] == 0u8);
    assert!(bytes[1] == 1u8);
    assert!(bytes[2] == 2u8);
    assert!(bytes[3] == 3u8);
    assert!(bytes[4] == 4u8);
    assert!(bytes[5] == 5u8);
}

...but is it reliable to assume that Vec<T> is represented as a single contiguous block of data this way?

The documentation on https://doc.rust-lang.org/std/vec/struct.Vec.html#capacity-and-reallocation says:

If a Vec has allocated memory, then the memory it points to is on the heap (as defined by the allocator Rust is configured to use by default), and its pointer points to len initialized, contiguous elements in order (what you would see if you coerced it to a slice), followed by capacity-len logically uninitialized, contiguous elements.

...but I'm not really sure if I understand what it means. Does this actually mean that for Vec<T> the underlying pointer is to a block of memory of length size_of::<T> * length of the Vec?

rust
1 Answers

Yes, a Vec<T> can be made into something that can be treated as a pointer to a block of memory of length std::mem::size_of::<T>() times the length of Vec.

There is one caveat, as what you are actually interested in is the slice of T, which the Vec can provide; the Vec itself should be considered an implementation detail. Besides that:

  1. A Vec<T> can deref to a slice [T]. Take that slice.

  2. The Rust Reference defines that a slice has the same layout as the section of the Array they slice. So when we deref from a Vec<T> to a [T], this slice of length n is guaranteed to have the same memory layout as an array [T; n].

  3. The Rust References defines the memory layout of an Array:

    Arrays are laid out so that the nth element of the array is offset from the start of the array by n * the size of the type bytes. An array of [T; n] has a size of size_of::<T>() * n and the same alignment of T.

  4. We know n (from [T]) and we know "the size of the type bytes" (via mem::size_of<T>()). Since all members of an array must be fully initialized at all times, and given the two sentences from the paragraph above, we know it is safe to access all bytes up until mem::size_of<T>() * length of Vec (actually length of slice, which introduces the array memory layout rule).

To make use of all that, you should make sure that you get a slice of the Vec first, use as_ptr() on the slice, and cast the raw pointer you get. This ensures the sequence of definitions as above. Your fn as_bytes(data: &[Point]) -> &[u8] is exactly correct.

user_27229680
Related