C# Passing struct without copying it to async function

I want to prevent the struct from being copied.

Is there a workaround?

unsafe struct Buffer
{
    public fixed byte buffer[40960000]; // big size buffer, sample
}

class Reader
{
    Reader()
    {
        var data = new Buffer();
        Read(ref data);     // <- No Copy, but not asynchronous.
        ReadAsync(data);    // <- Copy..
     // ReadAsync(ref data) <- impossible. but need.
    }

    void Read(ref Buffer data)
    {
        // Need asynchronous.
    }

    async void ReadAsync(Buffer data)
    {
        // Poor efficiency.
    }
}

This is code similar to my situation.

I want to efficiently handle structs that take up a lot of memory.