In F#, is it possible to pass a reference to a mutable, defaulted value as a parameter?

For the Froto project (Google Protobuf in F#), I am trying to update the deserialization code from using 'a ref objects to passing values byref<'a>, for performance.

However, the code below fails on the hydrator &element field line:

type Field = TypeA | TypeB | Etc

let hydrateRepeated
        (hydrator:byref<'a> -> Field -> unit)
        (result:byref<'a list>)
        (field:Field) =
    let mutable element = Unchecked.defaultof<'a>
    hydrator &element field
    result <- element :: result

error FS0421: The address of the variable 'element' cannot be used at this point

Is there anything I can do to get this code to work without changing the signature of the hydrator parameter?

I'm very aware that I could use hydrator:'a ref -> Field -> unit and get things to work. However, the goal is to support deserializing into record types without needing to create a bunch of ref objects on the heap every time a record is deserialize.

Note that the following code is perfectly legal and has the same signature as the hydrator function declaration, above, so I'm unclear on what the problem is.

let assign (result:byref<'a>) (x:'a) =
    result <- x

let thisWorks() =
    let mutable v = Unchecked.defaultof<int>
    assign &v 5
    printfn "%A" v