How do I get Newtonsoft to deserialize a date as it is and call it utc?

Viewed 4887

I have a datetime in a sql database: 2020-07-21 13:55:22.990 This datetime is in UTC although(correct me if I'm wrong), there would be no way to know that within the confines of the database. For reference, I am in CST(UTC-6) and this question was written during daylight savings time (so I'm -5 from UTC at the moment).

I'm sending a query to that database and getting back json using the following method (I don't believe there are any issues with it, but I've included it just in case):

private static string GetJSONFromSQLQuery(string query, string connectionString)
{
    var dataTable = new DataTable();

    using (var sqlConnection = new SqlConnection(connectionString))
    using (var sqlCommand = new SqlCommand(query, sqlConnection))
    using (var sqlDataAdapter = new SqlDataAdapter(sqlCommand))
    {
        sqlConnection.Open();
        sqlDataAdapter.Fill(dataTable);
    }

    var rowsAsDictionaries = new List<Dictionary<string, object>>();
    foreach (DataRow row in dataTable.Rows)
    {
        var rowAsDictionary = new Dictionary<string, object>();

        // ReSharper disable once LoopCanBeConvertedToQuery
        foreach (DataColumn column in dataTable.Columns)
            rowAsDictionary.Add(column.ColumnName, row[column]);

        rowsAsDictionaries.Add(rowAsDictionary);
    }

    return new JavaScriptSerializer().Serialize(rowsAsDictionaries);
}

This is a snippet of the json I get back (in a real call, there would be more properties in the object and generally more objects):


[
    {
        "Timestamp":"\\/Date(1595357722990)\\/",
    },
]

https://www.freeformatter.com/epoch-timestamp-to-date-converter.html says this is 7/21/2020, 1:55:22 PM under 'Convert epoch timestamp to date'

I then deserialize the json with newtonsoft:

var deserializedJson = JsonConvert.DeserializeObject<List<MyObjectType>>(json)

MyObjectType has a bunch of { get; set; } properties and one of them is:

public DateTime Timestamp { get; set; }

From the above json snippet and newtonsoft call, Timestamp would be set to 7/21, 6:55:22 and the Kind property would be set to Utc. This is incorrect not what I was expecting.

After some searching I learned about the DateTimeZoneHandling property in JsonSerializerSettings and made the following calls with each of the 4 enum options:

var local = JsonConvert.DeserializeObject<List<MyObjectType>>(json, new JsonSerializerSettings
{
    DateTimeZoneHandling = DateTimeZoneHandling.Local,

});

var roundtrip = JsonConvert.DeserializeObject<List<MyObjectType>>(json, new JsonSerializerSettings
{
    DateTimeZoneHandling = DateTimeZoneHandling.RoundtripKind,

});

var unspecified = JsonConvert.DeserializeObject<List<MyObjectType>>(json, new JsonSerializerSettings
{
    DateTimeZoneHandling = DateTimeZoneHandling.Unspecified,

});

var utc = JsonConvert.DeserializeObject<List<MyObjectType>>(json, new JsonSerializerSettings
{
    DateTimeZoneHandling = DateTimeZoneHandling.Utc,

});

Which gives me the following DateTime objects in MyObjectType (note these are snippets from a DateTime object):

deserializedJson:
    Time: 7/21 6:55:22,
    Kind: Utc

local:
    Time: 7/21 1:55:22,
    Kind: Local
    
roundtripKind:
    Time: 7/21 6:55:22,
    Kind: Utc
    
unspecified:
    Time: 7/21 6:55:22,
    Kind: Unspecified

utc:
    Time: 7/21 6:55:22,
    Kind: Utc

The Kind property is particularly important because I will be calling

TimeZoneInfo.ConvertTimeFromUtc(theDeserializedTimeStamp, TimeZoneInfo.Local);

My desired output is:

Time: 7/21 1:55:22,
Kind: Utc

Since that would be reflective of what is in the database and I would be able to call TimeZoneInfo.ConvertTimeFromUtc to convert the UTC time to whatever the local timezone is.

How do I get that output?

EDIT:

I found a workaround solution. I changed the return statement in GetJSONFromSQLQuery to:

return JsonConvert.SerializeObject(rowsAsDictionaries);

and all the DateTimeZoneHandling options left the time as 1:55 PM and had varying (but expected) results for the Kind property. Not sure why json from JavaScriptSerializer wouldn't play nicely, but I'm unstuck now at least.

The json that JsonConvert.SerializeObject looks like this:

[
   {
      "Timestamp":"2020-07-21T13:55:22.99"
   }
]
2 Answers

A few things:

const dt = new Date(1595357722990);
console.log(dt.toISOString());

  • You really should avoid JavaScriptSerializer. The docs say:

    ā“˜ Important

    For .NET Framework 4.7.2 and later versions, the APIs in the System.Text.Json namespace should be used for serialization and deserialization. For earlier versions of .NET Framework, use Newtonsoft.Json.

  • The format "\\/Date(1595357722990)\\/" is often refered to as an "ASP.NET JSON Date", because it was first released onto the world with early versions of ASP.Net, around the time that JSON starting becoming more prevalent than XML. It was a bad format then, and it is bad now. See On the nightmare that is JSON Dates... from Scott Hanselman back in 2012.

  • You should instead use the ISO 8601 extended date+time format (see also RFC 3339), which is the default for most modern JSON implementations, including Newtosoft JSON.Net and System.Text.Json.

  • You said:

    Timestamp would be set to 7/21, 6:55:22 and the Kind property would be set to Utc. This is incorrect.

    No, it is actually correct - assuming you meant PM. That's the value 1595357722990 represents. Probably you are creating that value incorrectly (see further below).

  • In most cases, including yours, you shouldn't need to change the default DateTimeZoneHandling option of JSON.Net.

  • You said your desired output is to have DateTimeKind.Utc - but it seems you meant after your conversion to local time? That doesn't make sense. If you are converting to local time, then the resulting kind would be DateTimeKind.Local.

  • You would be fine just using TimeZoneInfo.ConvertTime in this case. Since the input kind is already set to DateTimeKind.Utc, the ...FromUtc part isn't necessary. (That only really matters when the input kind is DateTimeKind.Unspecified.)

  • Looking at your edit, it seems you are actually storing 2020-07-21T13:55:22.99 in your database. If that field is intended to be interpreted as UTC, then indeed you should see hour 13 in the resulting JSON. However, it should also include a trailing Z to indicate that the data is UTC.

    Your JSON should actually look like this:

    [
        {
            "Timestamp":"2020-07-21T13:55:22.990Z"
        }
    ]
    

    It might have a few extra decimals places by default, which is fine.

  • The reason you aren't getting that result, and why the original value created by JavaScriptSerializer is wrong, is because you're not setting DateTimeKind.Utc on the values read from the database. You need to change your code to do that explicitly, using DateTime.SpecifyKind. For example, you could modify your inner loop to:

    foreach (DataColumn column in dataTable.Columns)
    {
        object o = row[column];
        if (o is DateTime)
        {
            o = DateTime.SpecifyKind((DateTime)o, DateTimeKind.Utc);
        }
    
        rowAsDictionary.Add(column.ColumnName, o);
    }
    
  • In general, you've got a lot going on in your code that is sub-optimal. Really you should be using an enumerable or list of a structured class instead of a dictionary of objects, and you should be using a DataReader rather than DataTable. Also, in most frameworks, such as in ASP.net, you should return that list directly and let the framework do the JSON conversion. In other words, you probably should not convert to JSON in this method at all.

It's been a while. Nobody else has given an answer and I've been using the solution I posted in the Edit for quite some time. So, I'm going to mark it as the answer.

Ultimately, the serializer I was using was old and not good. I thought I was using Newtonsoft, but I was wrong.

The method needs to be changed to this:

private static string GetJSONFromSQLQuery(string query, string connectionString)
{
    var dataTable = new DataTable();

    using (var sqlConnection = new SqlConnection(connectionString))
    using (var sqlCommand = new SqlCommand(query, sqlConnection))
    using (var sqlDataAdapter = new SqlDataAdapter(sqlCommand))
    {
        sqlConnection.Open();
        sqlDataAdapter.Fill(dataTable);
    }

    var rowsAsDictionaries = new List<Dictionary<string, object>>();
    foreach (DataRow row in dataTable.Rows)
    {
        var rowAsDictionary = new Dictionary<string, object>();

        // ReSharper disable once LoopCanBeConvertedToQuery
        foreach (DataColumn column in dataTable.Columns)
            rowAsDictionary.Add(column.ColumnName, row[column]);

        rowsAsDictionaries.Add(rowAsDictionary);
    }

    // The original return statement:
    //return new JavaScriptSerializer().Serialize(rowsAsDictionaries);

    // The correct return statement:
    return JsonConvert.SerializeObject(rowsAsDictionaries);
}

As PanagiotisKanavos said:

new JavaScriptSerializer() is not JSON.NET. It's an obsolete .NET serialised used as a stop-gap. It had a lot of quirks so people avoided it when possible. ASP.NET WebAPI and .NET Core up to 2.2 uses JSON.NET. .NET Core 3.1 uses a new JSON serialised that's even stricter



Also, a little out of scope, but if anyone wants an async method, this works:

private static async Task<string> GetJSONFromSQLQueryAsync(string query, string connectionString)
{
    var dataTable = new DataTable();

    using (var sqlConnection = new SqlConnection(connectionString))
    {
        await sqlConnection.OpenAsync();
        using var sqlCommand = new SqlCommand(query, sqlConnection);
        using var reader = await sqlCommand.ExecuteReaderAsync();

        dataTable.Load(reader);
    }

    var rowsAsDictionaries = new List<Dictionary<string, object>>();
    foreach (DataRow row in dataTable.Rows)
    {
        var rowAsDictionary = new Dictionary<string, object>();

        // ReSharper disable once LoopCanBeConvertedToQuery
        foreach (DataColumn column in dataTable.Columns)
            rowAsDictionary.Add(column.ColumnName, row[column]);

        rowsAsDictionaries.Add(rowAsDictionary);
    }

    return JsonConvert.SerializeObject(rowsAsDictionaries);
}
Related