Compress and serialize with rust: the specific case of `Vec<u8>`

Viewed 473

I am currently working on a way to compress data (structs I made) when serializing with serde. Everything works fine except with the specitic case of Vec. I'd like to know if some of you already met this problem or if you would have any thought to share :)

My goal is to provide a simple way of compressing any part of a struct by adding the #[serde(with="crate::compress")] macro. No matter of the underlying data structure, I want it to be compressed with my custom serialize function.

For instance, I want this structure to be serializable with compression :

struct MyCustomStruct {
    data: String,
    #[serde(with="crate::compress")]
    data2: SomeOtherStruct,
    #[serde(with="crate::compress")]
    data3: Vec<u8>,
}

For now, everything works fine and calls my custom module :

// in compress module

pub fn serialize<T, S>(data: T, serializer: S) -> Result(S::Ok, S::Error)
where
    T: Serialize,
    S: Serializer,
{
    // Simplified functioning below:
    let serialized_data: Vec<u8> = some_function(data);
    let compressed_data: Vec<u8> = some_other_fonction(serialized_data);
    Ok(choosen_serializer::serialize(compressed_data, serializer)?)
}

However, I do have a problem when it comes to compressing Vec elements (as data3 in the struct above).

Since the data is already Vec, I don't need to serialize it and I can directly pass it to my compression function. Worse: if I serialize it, I my not be with serde_bytes and so the compression call macro will result in increasing size!

I also don't want to have two different functions (one for Vec, one other) since it would be to the user to choose which one to use and using the wrong one with Vec would work but also increasing size (instead of decreasing it).

I already thought about / tried a few things but none of them can work :

1) Macro :

Very complicated since it needs to rewrite another macro. It would be a macro writing #[serde(with="crate::compress")] or #[serde(with="crate::compress_vec_u8")] depending on the type. I don't even know if this is possible, a meta-macro? :')

2) Trait implementation

It would be something like that :

trait CompressAndSerialize {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error)
    where S:Serializer;
}


impl CompressAndSerialize for Vec<u8> { ... }
impl<T> CompressAndSerialize for T where T:Serialize { ... }

but then I got an error (conflicting implementations of trait CompressAndSerialize for Vec) which seems normal since there are indeed two implementations for Vec :/

3) Worse solution but the one I'm heading for: using TypeId::of::<T>()

and skip serialization if data is already a Vec. I would still have to encapsulate it in an Enum so de-serialization will know if data is Vec or something else...

Edit: this isn't possible because type must have static lifetime which is almost always impossible (not in my case anyway)

Sorry this is a bit long and quite specific but I hope maybe one of you will have suggestions on how to deal with this problem :D

Edit: on the serde documentation (https://serde.rs/impl-serialize.html#other-special-cases) there is a link to a issue in rust-lang/rust (https://github.com/rust-lang/rust/issues/31844) and once this issue will be resolved I won't have any problem with the serialization of Vec since serde_bytes won't be needed anymore. Too bad this issue has been opened since Feb 2016 :'(

0 Answers
Related