An efficient way to deserialize the same json stream twice by Newtonsoft

user_1702804 0
Viewed 665

I have a json that I receive in a System.IO.StreamReader. Its structure is as follows:

{
    "data": [
        {
            "Property1": "Value1",
            "Property2": "Value2"
        },
        {
            "Property1": "Value3",
            "Property2": "Value4"
        }
    ],
    "metaData": {
        (some other structure)
    }
}

And I have two classes, one that has Property1 as a data field, and another one that has Property2 as a data field.

I would like to use the incoming stream to deserialize this json into two separate collections of the two classes that I have.

So far, I managed to do this only by stringifying the stream first and then invoking DeserliazeObject(string) twice. But then a huge chunk of memory is allocated unnecessarily.

How can I do this by means of a single invocation to DeserializeObject(streamReader)? Does Newtonsoft expose any serialization settings that can help me with this?

This is the code that I have, which is inefficient because I convert the stream to a string before invoking the deserializer:

var responseContent = await response.Content.ReadAsStringAsync();
var jObject = JObject.Parse(responseContent);
var records = GetRecords(jObject);

private IList<(IClass1, IClass2)> GetRecords(JObject jObject)
{
  return jObject["data"]
    .Children()
    .Select(CreateRecord)
    .ToList();
}

private (Class1, Class2) CreateRecord(JToken recordToken)
{
  var object1 = recordToken.ToObject<IClass1>();
  var object2 = recordToken.ToObject<IClass2>();
  return (object1, object2);
}
json.net c#
1 Answers

I offer you 3 paths, starting with the least efficient, but with the least code.

De-serialize into a JObject directly from Stream

This avoids converting the entire data into string first. Thus saving you the memory space for the string.

var serializer = new JsonSerializer();
using (var textReader = new StreamReader(stream))
using (var jsonReader = new JsonTextReader(textReader))
{
    var jobject = (JObject)serializer.Deserialize(jsonReader);
}

De-serialize into a C# class, then create the class you need. JObject is more heavy weight than POCO, consuming more memory.

This is straight forward. Suppose that the class you created is Data. 

var serializer = new JsonSerializer();
using (var textReader = new StreamReader(stream))
using (var jsonReader = new JsonTextReader(textReader))
{
    var data = serializer.Deserialize<Data>(jsonReader);
}

Use JsonTextReader to directly read into your target data structure

This involves manually reading the properties and values of your json stream into the structure you need. It saves the memory space of any intermediate objects (or rather, any intermediate objects will contain only the current token you are reading, not the entire data). If you return an IEnumerable<(IClass1, IClass2)>, the user of your method can actually process the result in a streaming fashion, making this an online algorithm.

I will leave you digging in the documentation to find out how to use it :)

user_50887930
Related