Is it possible to flatten Vec<&[u8]> into &[u8] without copying the data?

user_14923227 0
Viewed 375

I want to flatten a vector containing &[u8]:

fn main() {
    let x: &[u8] = &b"abc"[..];
    let y: &[u8] = &b"def"[..];
    
    let z: Vec<&[u8]> = vec![x, y];
    
    println!("z: {:?}", z);
    
    let z_flat: &[u8] = z.into_iter().flatten().collect();
    
    println!("z_flat: {:?}", z_flat);
}

I get the compiler error:

error[E0277]: a value of type `&[u8]` cannot be built from an iterator over elements of type `&u8`
 --> src/main.rs:9:49
  |
9 |     let z_flat: &[u8] = z.into_iter().flatten().collect();
  |                                                 ^^^^^^^ value of type `&[u8]` cannot be built from `std::iter::Iterator<Item=&u8>`
  |
  = help: the trait `FromIterator<&u8>` is not implemented for `&[u8]`

I have also tried the following:

let z_flat: &[u8] = &z.into_iter().map(|x| x.to_vec()).flatten().collect();

Compile error:

error[E0277]: a value of type `[u8]` cannot be built from an iterator over elements of type `u8`
 --> src/main.rs:9:70
  |
9 |     let z_flat: &[u8] = &z.into_iter().map(|x| x.to_vec()).flatten().collect();
  |                                                                      ^^^^^^^ value of type `[u8]` cannot be built from `std::iter::Iterator<Item=u8>`
  |
  = help: the trait `FromIterator<u8>` is not implemented for `[u8]`

The following does work (playground):

let z_flat: &[u8] = &z.concat();

I am not able to use concat in my real-world application because I am using another struct when retrieving the nested bytes (x and y) and this struct cannot implement Copy (only Clone) because it contains a Vec<u8> field. I have something similar to this (playground):

#[derive(Clone, Debug)]
struct Foo {
    Bar: Vec<u8>,
}

impl Foo {
    fn baz(&self) -> Vec<u8> {
        self.Bar
    }
}

fn main() {
    let x = Foo { Bar: vec![8u8] };
    let y = Foo { Bar: vec![18u8] };

    let z: Vec<Foo> = vec![x, y];

    println!("z: {:?}", z);

    let z_flat = z
        .into_iter()
        .map(|x| &x.baz()[..])
        .collect::<Vec<&[u8]>>()
        .concat();

    println!("z_flat: {:?}", z_flat);
}

Compile error:

error[E0507]: cannot move out of `self.Bar` which is behind a shared reference
 --> src/main.rs:8:9
  |
8 |         self.Bar
  |         ^^^^^^^^ move occurs because `self.Bar` has type `Vec<u8>`, which does not implement the `Copy` trait

error[E0515]: cannot return value referencing temporary value
  --> src/main.rs:22:18
   |
22 |         .map(|x| &x.baz()[..])
   |                  ^-------^^^^
   |                  ||
   |                  |temporary value created here
   |                  returns a value referencing data owned by the current function
rust
1 Answers

No, it is not possible. By definition, a slice is one contiguous region of memory. You don't have that to start with, so you can't create a slice.

Your workaround copies everything into one contiguous chunk of memory, which you can then get a slice of. I'd write it something like:

let z: Vec<&[u8]> = vec![b"abc", b"def"];
let z_alloc: Vec<u8> = z.iter().flat_map(|&x| x).copied().collect();
let z_flat: &[u8] = &z_alloc;

println!("z_flat: {:?}", z_flat);

For your larger example:

// fn baz(self) -> Vec<u8>
let z_flat: Vec<u8> = z.into_iter().flat_map(Foo::baz).collect();

// fn baz(&self) -> &[u8]
let z_flat: Vec<u8> = z.iter().flat_map(Foo::baz).copied().collect();

See also:

user_1554230
Related