Serialize and Deserialize CSV file as Stream

user_8417363 0
Viewed 2181

I'm trying to send a Stream of data to an API through my own API. The third-party API takes an object and the object has an object property who has a property value of Stream. In my code that consumes the my API, I need to read a CSV file and then serialize the DTO object which containts the Stream property to send it to my API. My API will then just pass the DTO object to the third party API.

My issue is that I am currently serializing the Stream property as a base 64 string, but am unable to de-serialize it back into the DTO object.

DTO Object:

public class BulkLeadRequest
{
    public Format3 FileFormat { get; set; }

    public FileParameter FileParameter { get; set; }

    public string LookupField { get; set; }

    public string PartitionName { get; set; }

    public int? ListId { get; set; }

    public int? BatchId { get; set; }
}

Format3 is an enum:

public enum Format3
{
    [System.Runtime.Serialization.EnumMember(Value = @"csv")]
    Csv = 0,

    [System.Runtime.Serialization.EnumMember(Value = @"tsv")]
    Tsv = 1,

    [System.Runtime.Serialization.EnumMember(Value = @"ssv")]
    Ssv = 2,
}

FileParamter object with Stream property:

public partial class FileParameter
{
    public FileParameter(System.IO.Stream data)
        : this(data, null)
    {
    }

    public FileParameter(System.IO.Stream data, string fileName)
        : this(data, fileName, null)
    {
    }

    public FileParameter(System.IO.Stream data, string fileName, string contentType)
    {
        Data = data;
        FileName = fileName;
        ContentType = contentType;
    }

    public System.IO.Stream Data { get; private set; }

    public string FileName { get; private set; }

    public string ContentType { get; private set; }
}

In order to test the serialization I have this small unit test.

public void TestSerializationDeserialization()
{
    BulkLeadRequest bulkLeadRequest = new BulkLeadRequest();
    bulkLeadRequest.FileFormat = Format3.Csv;
    bulkLeadRequest.FileParameter = new FileParameter(new StreamReader("C:\temp\\SmallFile.csv").BaseStream);

    string serializedObject = JsonConvert.SerializeObject(bulkLeadRequest, new StreamStringConverter());

    var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
        new JsonSerializerSettings {Converters = new List<JsonConverter> {new StreamStringConverter()}}); // Issue here

    Assert.Equal(bulkLeadRequest, obj);
}

The StreamStringConverter:

public class StreamStringConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return typeof(Stream).IsAssignableFrom(objectType);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        string objectContents = (string)reader.Value;
        byte[] base64Decoded = Convert.FromBase64String(objectContents);

        MemoryStream memoryStream = new MemoryStream(base64Decoded);

        return memoryStream;
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        FileStream valueStream = (FileStream)value;
        byte[] fileBytes = new byte[valueStream.Length];

        valueStream.Read(fileBytes, 0, (int)valueStream.Length);

        string bytesAsString = Convert.ToBase64String(fileBytes);

        writer.WriteValue(bytesAsString);
    }
}

In my TestSerializationDeserialization unit test, I get the following error:

An exception of type 'Newtonsoft.Json.JsonSerializationException' occurred in Newtonsoft.Json.dll but was not handled in user code Unable to find a constructor to use for type Cocc.MarketoSvcs.Business.FileParameter. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'FileParameter.Data', line 1, position 40.

If I add a default constructor to FileParameter then the Data property will be empty when deserialized. If I remove the implicit conversion in the JsonConvert.DeserializeObject<BulkLeadRequest>(..); call to JsonConvert.DeserializeObject(...); I will get an object, but not the BulkLeadRequest and with the base64 string still and not the Stream object I'd expect.

Serialized:

{"FileFormat":0,"FileParameter":{"Data":"RklSU1ROQU1FLE1ETElOSVQsTEFTVE5BTUUNCkNyaXN0aW5hLE0sRGlGYWJpbw0KTmVsbHksLFBhbGFjaW9zDQpNYXR0aGV3LEEsTmV2ZXJz","FileName":null,"ContentType":null},"LookupField":null,"PartitionName":null,"ListId":null,"BatchId":null}

What am I doing wrong?

deserialization json.net serialization c#
1 Answers

You are getting this error because the serializer does not know how to instantiate your FileParameter class. It prefers to use a default constructor if possible. It can use a parameterized constructor in some cases, if you give it a hint as to which one by marking it with a [JsonConstructor] attribute and all the constructor parameters match up to properties in the JSON object. However, that won't work in your case because all of your constructors expect a Stream and you require special handling for that.

To solve this you need a way for the serializer to instantiate your class without the stream and then use the converter to create the stream. You said you tried adding a default constructor, but then the stream was null. The reason it did not work is because all of your setters are private. With the default constructor, the serializer was able to instantiate the FileParameter, but then it could not populate it.

There are a couple of ways to make it work.

Solution 1 - modify the FileParameter class

  1. Add a default constructor to the FileParameter class as you did before. It can be private if you also mark it with [JsonConstructor].
  2. Add [JsonProperty] attributes to all of the private-set properties that need to be populated from the JSON. This will allow the serializer to write to the properties even though they are private.

So your class would look like this:

public partial class FileParameter
{
    public FileParameter(System.IO.Stream data)
        : this(data, null)
    {
    }

    public FileParameter(System.IO.Stream data, string fileName)
        : this(data, fileName, null)
    {
    }

    public FileParameter(System.IO.Stream data, string fileName, string contentType)
    {
        Data = data;
        FileName = fileName;
        ContentType = contentType;
    }

    [JsonConstructor]
    private FileParameter()
    {
    }

    [JsonProperty]
    public System.IO.Stream Data { get; private set; }

    [JsonProperty]
    public string FileName { get; private set; }

    [JsonProperty]
    public string ContentType { get; private set; }
}

Here is a working demo: https://dotnetfiddle.net/371ggK

Solution 2 - use a ContractResolver

If you cannot easily modify the FileParameter class because it is generated or you don't own the code, you can use a custom ContractResolver to do the same thing programmatically. Below is the code you would need for that.

public class CustomContractResolver : DefaultContractResolver
{
    protected override JsonObjectContract CreateObjectContract(Type objectType)
    {
        JsonObjectContract contract = base.CreateObjectContract(objectType);
        if (objectType == typeof(FileParameter))
        {
            // inject a "constructor" to use when creating FileParameter instances
            contract.DefaultCreator = () => new FileParameter(null);

            // make the private properties writable
            string[] propNames = new string[]
            {
                nameof(FileParameter.Data),
                nameof(FileParameter.FileName),
                nameof(FileParameter.ContentType),
            };
            foreach (JsonProperty prop in contract.Properties.Where(p => propNames.Contains(p.UnderlyingName)))
            {
                prop.Writable = true;
            }
        }
        return contract;
    }
}

To use the resolver, add it to the JsonSerializerSettings along with the StreamStringConverter:

var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
    new JsonSerializerSettings 
    { 
        ContractResolver = new CustomContractResolver(),
        Converters = new List<JsonConverter> { new StreamStringConverter() } 
    });

Working demo: https://dotnetfiddle.net/VHf359

By the way, in the WriteJson method of your StreamStringConverter, I noticed you are casting the value to a FileStream even though the CanConvert method says it can handle any Stream. You can fix it by changing this line:

FileStream valueStream = (FileStream)value;

to this:

Stream valueStream = (Stream)value;
user_102630
Related