How to create a smart pointer to an unsized type with an embedded slice?

user_12987856 0
Viewed 518

I'm trying to avoid multiple heap allocations by using something like C's flexible-array member. For that, I need to allocate an unsized struct, but I didn't find any way to do that through smart pointers. I'm specifically interested in Rc, but this is also the case for Box, so that's what I'll use in the example.

Here is the closest I've gotten so far:

use std::alloc::{self, Layout};

struct Inner {/* Sized fields */}

#[repr(C)] // Ensure the array is always last
// Both `inner` and `arr` need to be allocated, but preferably not separately
struct Unsized {
    inner: Inner,
    arr: [usize],
}

pub struct Exposed(Box<Unsized>);

impl Exposed {
    pub fn new(capacity: usize) -> Self {
        // Create a layout of an `Inner` followed by the array
        let (layout, arr_base) = Layout::array::<usize>(capacity)
            .and_then(|arr_layout| Layout::new::<Inner>().extend(arr_layout))
            .unwrap();
        let ptr = unsafe { alloc::alloc(layout) };
        // At this point, `ptr` is `*mut u8` and the compiler doesn't know the size of the allocation
        if ptr.is_null() {
            panic!("Internal allocation error");
        }
        unsafe {
            ptr.cast::<Inner>()
                .write(Inner {/* Initialize sized fields */});
            let tmp_ptr = ptr.add(arr_base).cast::<usize>();
            // Initialize the array elements, in this case to 0
            (0..capacity).for_each(|i| tmp_ptr.add(i).write(0));
            // At this point everything is initialized and can safely be converted to `Box`
            Self(Box::from_raw(ptr as *mut _))
        }
    }
}

This doesn't compile:

error[E0607]: cannot cast thin pointer `*mut u8` to fat pointer `*mut Unsized`
  --> src/lib.rs:32:28
   |
32 |         Self(Box::from_raw(ptr as *mut _))
   |                            ^^^^^^^^^^^^^

I could work directly with *mut u8, but that seems extremely error-prone and requires manual dropping.

Is there a way to create a fat pointer from ptr, since I actually know the allocation size, or to create a smart pointer from a compound unsized type?

rust smart-pointers
1 Answers

The problem is that the pointer *mut Unsized is a wide pointer, so not just an address, but an address and the number of elements in the slice. The pointer *mut u8 on the other hand contains no information about the length of the slice. The standard library supplies the

  • std::ptr::slice_from_raw_parts and,
  • std::ptr::slice_from_raw_parts_mut

for this situtation. So you first create a fake (and wrong) *mut usize

ptr as *mut usize

which then allows

slice_from_raw_parts_mut(ptr as *mut usize, capacity)

to create a fake (and still wrong) *mut [usize] with the correct length field in the wide pointer, which we then unceremonously cast

slice_from_raw_parts_mut(ptr as *mut usize, capacity) as *mut Unsized

which does nothing but change the type (the value is unchanged), so we get the correct pointer that we can now finally feed into Box::from_raw

Full example demonstrating this post:

use std::alloc::{self, Layout};

struct Inner {/* Sized fields */}

#[repr(C)] // Ensure the array is always last
           // Both `inner` and `arr` need to be allocated, but preferably not separately
struct Unsized {
    inner: Inner,
    arr: [usize],
}

pub struct Exposed(Box<Unsized>);

impl Exposed {
    pub fn new(capacity: usize) -> Self {
        // Create a layout of an `Inner` followed by the array
        let (layout, arr_base) = Layout::array::<usize>(capacity)
            .and_then(|arr_layout| Layout::new::<Inner>().extend(arr_layout))
            .unwrap();
        let ptr = unsafe { alloc::alloc(layout) };
        // At this point, `ptr` is `*mut u8` and the compiler doesn't know the size of the allocation
        if ptr.is_null() {
            panic!("Internal allocation error");
        }
        unsafe {
            ptr.cast::<Inner>()
                .write(Inner {/* Initialize sized fields */});
            let tmp_ptr = ptr.add(arr_base).cast::<usize>();
            // Initialize the array elements, in this case to 0
            (0..capacity).for_each(|i| tmp_ptr.add(i).write(0));
        }

        // At this point everything is initialized and can safely be converted to `Box`
        unsafe {
            Self(Box::from_raw(
                std::ptr::slice_from_raw_parts_mut(ptr as *mut usize, capacity) as *mut Unsized,
            ))
        }
    }
}

Playground

Side note: You do not need #[repr(C)] to make sure that the unsized slice field is at the end, that is guaranteed. What you need it for is knowing the offsets to the fields.

user_11036810
Related