Why is a lifetime needed when implementing From<&[u8]>

user_338097 0
Viewed 132

I'm trying to have a MyType that supports a From<&[u8]> trait, but I'm running into "lifetime problems":

Here's a minimally viable example:

struct MyType {
    i: i32
}

impl MyType {
    fn from_bytes(_buf: &[u8]) -> MyType {
        // for example...
        MyType { i: 3 }
    }
}

impl From<&[u8]> for MyType {
    fn from(bytes: &[u8]) -> Self {
        MyType::from_bytes(bytes)
    }
}

fn do_smth<T>() -> T where T: From<&[u8]>
{
    // for example...
    let buf : Vec<u8> = vec![1u8, 2u8];
    T::from(buf.as_slice())
}

(...and here's a Rust playground link)

For reasons I cannot understand, the Rust compiler is telling me:

error[E0637]: `&` without an explicit lifetime name cannot be used here
  --> src/lib.rs:17:36
   |
17 | fn do_smth<T>() -> T where T: From<&[u8]>
   |                                    ^ explicit lifetime name needed here

I'm not an expert on lifetimes and I don't understand why this piece of code needs one. What would be the best way to fix this?

Might Rust be thinking that the type T could be a &[u8] itself? But, in that case, the lifetime should be inferred to be the same as the input to From::<&[u8]>::from(), no?

One fix I was given was to do:

fn do_smth<T>() -> T where for<'a> T: From<&'a [u8]>
{
    // for example...
    let buf : Vec<u8> = vec![1u8, 2u8];
    T::from(buf.as_slice())
}

...but I do not understand this fix, nor do I understand why lifetimes are needed in the first place.

rust
2 Answers

Rust first wants you to write:

fn do_smth<'a, T>() -> T
where
    T: From<&'a [u8]>,
{
    // for example...
    let buf: Vec<u8> = vec![1u8, 2u8];
    T::from(&buf)
}

where you make explicit that this function can be called for any lifetime 'a and any type T such that T implements From<&'a [u8]>.

But Rust then complains:

error[E0597]: `buf` does not live long enough
  --> src/lib.rs:24:13
   |
18 | fn do_smth<'a, T>() -> T
   |            -- lifetime `'a` defined here
...
24 |     T::from(&buf)
   |     --------^^^^-
   |     |       |
   |     |       borrowed value does not live long enough
   |     argument requires that `buf` is borrowed for `'a`
25 | }
   | - `buf` dropped here while still borrowed

You promised that this function could work with any lifetime, but this turns out to not be true, because in the body of the function you create a fresh reference to the local Vec which has a different lifetime, say 'local. Your function only works when 'a equals 'local, but you promise that it also works for all other lifetimes. What you need is a way to express that these lifetimes are the same, and the only way I think that is possible is by changing the local reference to an argument:

fn do_smth<'a, T>(buf: &'a [u8]) -> T
where
    T: From<&'a [u8]>,
{
    T::from(buf)
}

And then it compiles.

If instead of the function promising it can work with any lifetime, you want to make the caller promise it can work with any lifetime, you can instead use HRTBs to make the caller promise it.

fn do_smth<T>() -> T
where
    for<'a> T: From<&'a [u8]>,
{
    // for example...
    let buf: Vec<u8> = vec![1u8, 2u8];
    T::from(&buf)
}

Now, since you can use any lifetime, a local one also works and the code compiles.

user_56003630

Lifetimes represent a "duration" (metaphorically), or, more pragmatically, a scope, in which a variable is valid. Outside of one's lifetime, the variable should be considered as having been freed from memory, even though you haven't done that explicitly, because that's how Rust manages memory.

It becomes a bit more complex when Rust tries to ensure that, once a variable is done for, no other parts of the code that could have had access to that variable still have access. These shared accesses are called borrows, and that's why borrows have lifetimes too. The main condition Rust enforces on them is that a borrow's lifetime is always shorter (or within, depending on how you see it) than its original variable, ie. you can't share something for more time than you actually own it.

Rust therefore enforces all borrows (as well as all variables, really) to have an established lifetime at compile-time. To lighten things, Rust has default rules about what a lifetime should be if it was not explicitly defined by the user, that is, when you talk about a type that involves a lifetime, Rust let's you not write that lifetime explicitly under certain conditions. However, this is not a "lifetime inference", in the sense of inferring types: Rust will not try to make sense out of explicit lifetimes, it's a lot less smart about it. In particular, this lifetime explicitation can fail, in the sense that Rust will not be able to figure out the right lifetime it has to assign even though it was possible to find out that worked.

Back to business: your first error simply stems from the fact that Rust has no rule to make a lifetime if it wasn't provided in the position pointed out by the error. As I said, Rust won't try to infer what the right lifetime would be, it simply checks whether not explicitly putting a lifetime there implicitly means something or not. So, you simply need to put one.

Your first reflex might be to make your function generic over the missing lifetime, which is often the right thing to do (and even the only possible action), that is, do something like that:

fn do_smth<'a, T>() -> T 
where
   T: From<&'a [u8]>
{
    // for example...
    let buf : Vec<u8> = vec![1, 2];
    T::from(buf.as_slice())
}

What this means is that do_smth is generic over the lifetime 'a, just like it is generic over the type T. This has two consequences:

  • Rust will proceed to a monomorphisation of your function for each call, meaning it will actually provide a concrete implementation of your function for each type T and each lifetime 'a that is required. In particular, it will automatically find out what is the right lifetime. This might seem contradictory with what I said earlier, about Rust not inferring lifetimes. The difference is that type inference and monomorphisation, although similar, are not the same step, and so the compiler does not work lifetimes in the same way. Don't worry about this until you have understood the rest.
  • The second consequence, which is a bit disastrous, is that your function exposes the following contract: for any type T, and any lifetime 'a, such that T: From<&'a [u8]>, do_smth can produce a type T. If you think about it, it means that even if T only implements From<&'a [u8]> for a lifetime 'a that is already finished (or, if you see lifetimes as scopes, for a lifetime 'a that is disjoint from do_smth's scope), you can produce an element of type T. This is not what you actually meant: you don't want the caller to give you an arbitrary lifetime. Instead, you know that the lifetime of the borrow of the slice is the one you chose it to be, within your function (because you own the underlying vector), and you want that the type T to be buildable from that slice. That is, you want T: From<&'a [u8]> for a 'a that you have chosen, not one provided by the caller.

This last point should make you understand why the previous snippet of code is unsound, and won't compile. Your function should not take a lifetime as argument, just a type T with certain constraints. But then, how do you encode the said conditions? That's where for<'a> comes into play. If you have a type T such that T: for<'a> From<&'a [u8]>, it means that for all 'a, T: From<&'a [u8]>. In particular, it is true for the lifetime of your slice. This is why the following works

fn do_smth<T>() -> T
where
    T: for<'a> From<&'a [u8]>
{
    // for example...
    let buf: Vec<u8> = vec![1, 2];
    T::from(buf.as_slice())
}

Note that, as planned, this version of do_smth is not generic over a lifetime, that is, the caller does not provide a lifetime to the function.

user_59562610
Related