Transpose data from records to columns without intermediate struct using serde Rust

user_15025374 0
Viewed 142

I have data structured in two different ways:

1: JSON

{
    "key1": 40,
    "key2": 50
{,
{
    "key1": 41,
    "key2": 51
}

2: nested array

[[40,50],[41,51]]

The goal is to deserialize this data (I receive both as Strings) into an Struct that looks like this:

struct data {
    key1: Vec<i8>,      // -> [40,41]
    key2: Vec<i8>       // -> [50,51]
}

I already have 2 methods for deserializing each type of data, but the problem is that for the first one, I have to create an intermediate Struct and collect them in a Vec to then iterate over this Vec pushing each element to its specific Vecs in the final Struct. For the second one, I deserialize to a Vec<Vec<i8>> to then iterate again transposing into the final Struct element by element.

I read through all serde documentation and tried to find examples, but couldn't find a way of pushing directly to the final Vecs of the Struct without an intermediate step. Does serde supports this? If so, how it's implemented?

rust deserialization struct json
1 Answers

To do this you need a custom visitor for your array. Below is a working implementation.

Note that although we use additional enum InnerData, it does not require any additional allocations as this struct is only used in the stack. For every element of the outer array InnerData will be deserialized, and its fields pushed to the fields of the Data struct.

#[serde(untagged)] allows deserialization of the enum from flat variants (without specifying Map or Array in json).

Also not, that to use this type of deserialization, you need to specifically let the Deserializer know which Visitor to use. If your stuct is one of the feilds of another structure, you can specify this with #[serde(deserialize_with = ...)] attribute.

use serde::de;
use serde::de::Deserializer;
use serde::Deserialize;

const A: &str = "[{\"key1\": 40, \"key2\": 50}, {\"key1\": 41, \"key2\": 51}]";
const B: &str = "[[40, 50], [41, 51]]";

#[derive(Debug, Deserialize)]
struct Data {
    key1: Vec<i8>,
    key2: Vec<i8>,
}

#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum InnerData {
    Map { key1: i8, key2: i8 },
    Array(i8, i8),
}

struct DataVisitor;

impl<'de> de::Visitor<'de> for DataVisitor {
    type Value = Data;

    fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(formatter, "invalid input")
    }

    fn visit_seq<A: de::SeqAccess<'de>>(self, mut seq: A) -> Result<Self::Value, A::Error> {
        let mut result = Data {
            key1: vec![],
            key2: vec![],
        };
        while let Some(inner) = seq.next_element::<InnerData>()? {
            let (k1, k2) = match inner {
                InnerData::Map { key1, key2 } => (key1, key2),
                InnerData::Array(key1, key2) => (key1, key2),
            };

            result.key1.push(k1);
            result.key2.push(k2);
        }

        Ok(result)
    }
}

fn main() {
    let mut deserializer = serde_json::Deserializer::from_str(A);
    println!("{:?}", deserializer.deserialize_seq(DataVisitor {}));

    let mut deserializer = serde_json::Deserializer::from_str(B);
    println!("{:?}", deserializer.deserialize_seq(DataVisitor {}));
}

Fiddle: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f921adbb490970eb9a8b5ef5f9ab49d0

user_49978790
Related