How to implement Serde Deserializer for wire protocol with multiple representations of strings?

user_16277078 0
Viewed 231

I am trying to implement a Serde Serializer/Deserializer for the Kafka wire protocol. In this protocol, there are 4 different string representations. This poses a problem when implementing the deserializer: when deserialize_str is called to attempt to deserialize a string for a given message, there’s no way to know whether a string starts with i32 or a varint, since neither the deserialize_str method or the provided visitor provides any kind of metadata or type information that could be used to help make this decision.

My first thought was that I could implement a new type wrapper and use a custom deserialize implementation, but I now understand this doesn’t make sense because it needs to be generic over all deserializers, not just the deserializer I’m building. The wrapper still just asks the Deserializer to read a string.

I’m struggling to come up with a good solution here, and can’t find examples of other data formats that use multiple representations for a given data type.

serde rust
1 Answers

Here's the EXTREMELY HACKY way that I'm planning on solving this using a new type wrapper and custom Deserialize implementation. More specifically, my Deserialize impl will have it's own Visitor and we're going abuse Visitor here.

First I was hoping that I could use TypeId on Visitor::Value, but Visitor doesn't have a static lifetime, so that doesn't work.

Instead, I'm using the "expected" error message as a type tag, using the serde::de::Expected trait:

 fn deserialize_str<V>(self, visitor: V) -> Result<V::Value>
        where
            V: Visitor<'de>,
    {
        use serde::de::Expected;
        let ty = format!("{}", &visitor as &dyn Expected);
        if ty == "NullableString" {
          // ... do nullable dstring deserialization logic here
        }
      // ...
    }

This... works! But feels extremely gross. Not totally satisfied, but goign to proceed with this for now.

user_162770780
Related