Is it possible to safely serialize an unmanaged type?

Viewed 64

When I found out about unmanaged generic constraint, I was really excited because I thought this would solve my serialization needs. However, reading more about it, it looks like using the constraint for serialization purposes is not a good idea.

For context, I am creating a low-level networking library and I want to allow the user to easily serialize simple types, such as primitives or enums. I created a generic method that takes in any unmanaged type and writes it to a byte-array.

public unsafe void Write<T>(T value, int offset) where T : unmanaged
{
    fixed (byte* pointer = &Bytes[offset])
    {
        *(T*) pointer = value;
    }
}

Reading is just as simple:

public unsafe T Read<T>(int offset) where T : unmanaged
{
    fixed (byte* pointer = &Bytes[offset])
    {
        return *(T*) pointer;
    }
}

At first glance, these methods are perfect for serialization; they are extremely fast (simple direct memory write or read), they are generic (no need to write methods for each primitive such as int, float etc.) and they even support user defined types out of the box (enums and custom unmanaged structs).

But as usual, when something looks too good to be true, it usually is. First problem is that these methods won't work if 2 systems use different endianness. For example, system 1 writes an integer, sends it over the network where system 2 (with different endianness) tries to read it. System 2 is going to interpret a wrong value.

Second potential problem is struct layout where different platforms might have different memory layouts of structs, so we have the same problem as with endianness.

This makes me wonder whether generic serialization that would work on all platforms, regardless of endianness or struct layout, is even possible. If so, I'd love to see the solution. Or is the best (and only) solution to write serialization methods for each type separately, as can be seen in classes such as BitConverter, BinaryWriter, BinaryReader and relatively new BinaryPrimitives?

1 Answers

Yes, you can use this feature to speed up the code in certain cases, if you are careful.

  1. Layout. I would not say it is a problem. The network protocol should clearly define the data format. Then we implement types accordingly, using [StructLayout(LayoutKind.Sequential, Pack=...)] (or equivalent in other languages).

  2. Endianness. Again, the network protocol has specific endianness. If protocol endianness matches current hardware endianness, we can use direct copy method like described in the question. Otherwise, we should write fields individually, reversing byte order (with BinaryPrimitives.ReverseEndianness).

As a maintainer of a custom binary protocol, I'd recommend against inventing your own protocol. It is hard and error-prone.

Just take Avro or MessagePack - those are great.

Related