Performant Entity Serialization: BSON vs MessagePack (vs JSON)

A key difference not yet mentioned is that BSON contains size information in bytes for the entire document and further nested sub-documents.

document    ::=     int32 e_list

This has two major benefits for restricted environments (e.g. embedded) where size and performance is important.

1. You can immediately check if the data you're going to parse represents a complete document or if you're going to need to request more at some point (be it from some connection or storage). Since this is most likely an asynchronous operation you might already send a new request before parsing.
2. Your data might contain entire sub-documents with irrelevant information for you. BSON allows you to easily traverse to the next object past the sub-document by using the size information of the sub-document to skip it. msgpack on the other hands contains the number of elements inside whats called a map (similar to BSON's sub-documents). While this is undoubtedly useful information it doesn't help the parser. You'd still have to parse every single object inside the map and can't just skip it. Depending on the structure of your data this might have a huge impact on performance.

Well，as the author said，MessagePack is originally designed for network communication while BSON is designed for storages.

MessagePack is compact while BSON is verbose. MessagePack is meant to be space-efficient while BSON is designed for CURD (time-efficient).

Most importantly, MessagePack's type system (prefix) follow Huffman encoding, here I drawed a Huffman tree of MessagePack(click link to see image)：

I made quick benchmark to compare encoding and decoding speed of MessagePack vs BSON. BSON is faster at least if you have large binary arrays:

BSON writer: 2296 ms (243487 bytes)
BSON reader: 435 ms
MESSAGEPACK writer: 5472 ms (243510 bytes)
MESSAGEPACK reader: 1364 ms

Using C# Newtonsoft.Json and MessagePack by neuecc:

    public class TestData
    {
        public byte[] buffer;
        public bool foobar;
        public int x, y, w, h;
    }

    static void Main(string[] args)
    {
        try
        {
            int loop = 10000;

            var buffer = new TestData();
            TestData data2;
            byte[] data = null;
            int val = 0, val2 = 0, val3 = 0;

            buffer.buffer = new byte[243432];

            var sw = new Stopwatch();

            sw.Start();
            for (int i = 0; i < loop; i++)
            {
                data = SerializeBson(buffer);
                val2 = data.Length;
            }

            var rc1 = sw.ElapsedMilliseconds;

            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                data2 = DeserializeBson(data);
                val += data2.buffer[0];
            }
            var rc2 = sw.ElapsedMilliseconds;

            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                data = SerializeMP(buffer);
                val3 = data.Length;
                val += data[0];
            }

            var rc3 = sw.ElapsedMilliseconds;

            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                data2 = DeserializeMP(data);
                val += data2.buffer[0];
            }
            var rc4 = sw.ElapsedMilliseconds;

            Console.WriteLine("Results:", val);
            Console.WriteLine("BSON writer: {0} ms ({1} bytes)", rc1, val2);
            Console.WriteLine("BSON reader: {0} ms", rc2);
            Console.WriteLine("MESSAGEPACK writer: {0} ms ({1} bytes)", rc3, val3);
            Console.WriteLine("MESSAGEPACK reader: {0} ms", rc4);
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }

        Console.ReadLine();
    }

    static private byte[] SerializeBson(TestData data)
    {
        var ms = new MemoryStream();

        using (var writer = new Newtonsoft.Json.Bson.BsonWriter(ms))
        {
            var s = new Newtonsoft.Json.JsonSerializer();
            s.Serialize(writer, data);
            return ms.ToArray();
        }
    }

    static private TestData DeserializeBson(byte[] data)
    {
        var ms = new MemoryStream(data);

        using (var reader = new Newtonsoft.Json.Bson.BsonReader(ms))
        {
            var s = new Newtonsoft.Json.JsonSerializer();
            return s.Deserialize<TestData>(reader);
        }
    }

    static private byte[] SerializeMP(TestData data)
    {
        return MessagePackSerializer.Typeless.Serialize(data);
    }

    static private TestData DeserializeMP(byte[] data)
    {
        return (TestData)MessagePackSerializer.Typeless.Deserialize(data);
    }