Serde struct version check

user_6094756 0
Viewed 441

I want to add a simple version scheme + check to my struct:

#[derive(Serialize, Deserialize)]
struct Versioned {
    version: u32,
    other_field: String,
}

impl Versioned {
    const FORMAT_VERSION: u32 = 42;
}

The idea is rather simple: the version field is deserialized (0 if not present*), and an error is thrown if it does not equal FORMAT_VERSION. Then, the rest of the struct is processed. The order here is important: otherwise, the deserialization error might not be correct (you might get something like "missing field new_field" instead of "version too old").

I've tried a few ideas with wrapper types and custom deserializers, but none of them provided a good user experience. My best so far:

#[derive(Serialize, Deserialize)]
struct Payload {
    some_field: String,
}

#[derive(Serialize, Deserialize)]
#[serde(tag = "version")]
enum Versioned {
    #[serde(rename = "0")]
    Inner(Payload),
}

It uses an enum wrapper for the version check, and two From implementations for easy type conversion. The downsides are:

  • No default value for the field* (= not backwards compatible with unversioned data)
  • Bad error message on old version ("Error: unknown variant 42, expected 0 at line 2 column 17" instead of something like "invalid version")

* Notice: the backwards compatibility requirement makes things significantly more difficult. So remember folks, always add a version field to all your data right away!

serde rust
3 Answers

My first approach was to try and build a custom deserializer which first deserializes into a new struct (VersionedModel which derives Deserialize) with only the version field. This doesn't work since VersionedModel::deserialize(deserializer) consumes the deserializer so you're unable to deserialize any other data.

My second approach would be using one of the methods on Deserializer<'de>, however, all methods consume the deserializer, so you wouldn't be able to first read the version and then the other fields.

The third (and most naive approach, not so clean but it works) would be to to implement your deserialize in 2 steps like this:

use serde::{Deserialize, Serialize};
use thiserror::Error;

#[derive(Deserialize, Serialize)]
struct Versioned {
    version: u32,
    other_field: String,
}

impl Versioned {
    const FORMAT_VERSION: u32 = 42;
}

#[derive(Debug, Error)]
pub enum DeserializeError {
    #[error("Version field is missing")]
    VersionIsMissing,
    #[error("Invalid version, got: {actual:}, expected: {expected:}")]
    InvalidVersion { actual: u32, expected: u32 },
    #[error(transparent)]
    DeserializeError(#[from] serde_json::Error),
}

fn deserialize_model(value: &str) -> Result<Versioned, DeserializeError> {
    #[derive(Deserialize, Serialize)]
    struct VersionOnly {
        version: u32,
    }

    let VersionOnly { version } =
        serde_json::from_str(&value).map_err(|_| DeserializeError::VersionIsMissing)?;

    if version == Versioned::FORMAT_VERSION {
        Ok(serde_json::from_str(&value)?)
    } else {
        Err(DeserializeError::InvalidVersion {
            actual: version,
            expected: Versioned::FORMAT_VERSION,
        })
    }
}

Playground

Last approach: write your own serializer, write your own visitors (basically copy paste all generated macro code and add some changes). I would not recommend this approach since it's very error prone and probably not worth the trouble.

user_32579010

You can achieve both your points by writing a custom deserialization function for the version field. You make it optional by putting #[serde(default)] on the field. The custom deserialization function can have any logic you want for checking the version number and produce your desired error message.

#[derive(Serialize, Deserialize, Debug)]
struct Versioned {
    #[serde(deserialize_with = "ensure_version", default)]
    version: u32,
    other_field: String,
}

impl Versioned {
    const FORMAT_VERSION: u32 = 42;
}

fn ensure_version<'de, D: Deserializer<'de>>(d: D) -> Result<u32, D::Error> {
    let version = u32::deserialize(d)?;
    if version != Versioned::FORMAT_VERSION {
        return Err(D::Error::custom("Version mismatch for Versioned"));
    }
    Ok(version)
}

Playground

user_154702860

Thanks to all the other answers for the inspiration! See also https://github.com/serde-rs/serde/issues/1799 and https://github.com/serde-rs/serde/issues/912#issuecomment-981657911 for more details on the untagged enum trick I'm using.

Basically, two steps are needed to improve the code from the question as desired:

  1. Add a #[serde(other)] Unsupported variant to the versioned enum
  2. Wrap the enum in another, untagged enum that handles the default version (called Legacy)
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, PartialEq, Debug)]
pub struct Data {
    field: String,
}

#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(tag = "version")]
pub enum VersionChecked {
    #[serde(rename = "0")]
    V0(Data),
    #[serde(other)]
    Unsupported,
}

#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum DefaultVersion {
    Versioned(VersionChecked),
    Legacy(Data),
    // Optional, if you want to "catch" the serde error
    // Invalid(serde_json::Value),
}

fn main() {
    dbg!(serde_json::from_str::<DefaultVersion>(r#"{ "version": 0, "field": "Somestring" }"#).unwrap());
    dbg!(serde_json::from_str::<DefaultVersion>(r#"{ "field": "Somestring" }"#).unwrap());
    dbg!(serde_json::from_str::<DefaultVersion>(r#"{ "blablah": "blablah" }"#));
    dbg!(serde_json::from_str::<DefaultVersion>(r#"{ "version": 42, "foo": "bar" }"#));
}

Playground

Some improvements on the usage can still be made. I'm thinking of TryFrom<DefaultVersion> for Data for the one direction, and some serialize_as for the other one. But for now it will do.

user_60947560
Related