How to overcome "datetime.datetime not JSON serializable"?

if you are using python3.7, then the best solution is using datetime.isoformat() and datetime.fromisoformat(); they work with both naive and aware datetime objects:

#!/usr/bin/env python3.7

from datetime import datetime
from datetime import timezone
from datetime import timedelta
import json

def default(obj):
    if isinstance(obj, datetime):
        return { '_isoformat': obj.isoformat() }
    raise TypeError('...')

def object_hook(obj):
    _isoformat = obj.get('_isoformat')
    if _isoformat is not None:
        return datetime.fromisoformat(_isoformat)
    return obj

if __name__ == '__main__':
    #d = { 'now': datetime(2000, 1, 1) }
    d = { 'now': datetime(2000, 1, 1, tzinfo=timezone(timedelta(hours=-8))) }
    s = json.dumps(d, default=default)
    print(s)
    print(d == json.loads(s, object_hook=object_hook))

output:

{"now": {"_isoformat": "2000-01-01T00:00:00-08:00"}}
True

if you are using python3.6 or below, and you only care about the time value (not the timezone), then you can use datetime.timestamp() and datetime.fromtimestamp() instead;

if you are using python3.6 or below, and you do care about the timezone, then you can get it via datetime.tzinfo, but you have to serialize this field by yourself; the easiest way to do this is to add another field _tzinfo in the serialized object;

finally, beware of precisions in all these examples;

You should apply .strftime() method on .datetime.now() method to make it as a serializable method.

Here's an example:

from datetime import datetime

time_dict = {'time': datetime.now().strftime('%Y-%m-%dT%H:%M:%S')}
sample_dict = {'a': 1, 'b': 2}
sample_dict.update(time_dict)
sample_dict

Output:

Out[0]: {'a': 1, 'b': 2, 'time': '2017-10-31T15:16:30'}

The json.dumps method can accept an optional parameter called default which is expected to be a function. Every time JSON tries to convert a value it does not know how to convert it will call the function we passed to it. The function will receive the object in question, and it is expected to return the JSON representation of the object.

def myconverter(o):
 if isinstance(o, datetime.datetime):
    return o.__str__()

print(json.dumps(d, default = myconverter)) 

Actually it is quite simple. If you need to often serialize dates, then work with them as strings. You can easily convert them back as datetime objects if needed.

If you need to work mostly as datetime objects, then convert them as strings before serializing.

import json, datetime

date = str(datetime.datetime.now())
print(json.dumps(date))
"2018-12-01 15:44:34.409085"
print(type(date))
<class 'str'>

datetime_obj = datetime.datetime.strptime(date, '%Y-%m-%d %H:%M:%S.%f')
print(datetime_obj)
2018-12-01 15:44:34.409085
print(type(datetime_obj))
<class 'datetime.datetime'>

As you can see, the output is the same in both cases. Only the type is different.

Generally there are several ways to serialize datetimes, like:

1. ISO string, short and can include timezone info, e.g. @jgbarah's answer
2. Timestamp (timezone data is lost), e.g. @JayTaylor's answer
3. Dictionary of properties (including timezone).

If you're okay with the last way, the json_tricks package handles dates, times and datetimes including timezones.

from datetime import datetime
from json_tricks import dumps
foo = {'title': 'String', 'datetime': datetime(2012, 8, 8, 21, 46, 24, 862000)}
dumps(foo)

which gives:

{"title": "String", "datetime": {"__datetime__": null, "year": 2012, "month": 8, "day": 8, "hour": 21, "minute": 46, "second": 24, "microsecond": 862000}}

So all you need to do is

`pip install json_tricks`

and then import from json_tricks instead of json.

The advantage of not storing it as a single string, int or float comes when decoding: if you encounter just a string or especially int or float, you need to know something about the data to know if it's a datetime. As a dict, you can store metadata so it can be decoded automatically, which is what json_tricks does for you. It's also easily editable for humans.

Disclaimer: it's made by me. Because I had the same problem.

Here is my full solution for converting datetime to JSON and back..

import calendar, datetime, json

def outputJSON(obj):
    """Default JSON serializer."""

    if isinstance(obj, datetime.datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()

        return obj.strftime('%Y-%m-%d %H:%M:%S.%f')
    return str(obj)

def inputJSON(obj):
    newDic = {}

    for key in obj:
        try:
            if float(key) == int(float(key)):
                newKey = int(key)
            else:
                newKey = float(key)

            newDic[newKey] = obj[key]
            continue
        except ValueError:
            pass

        try:
            newDic[str(key)] = datetime.datetime.strptime(obj[key], '%Y-%m-%d %H:%M:%S.%f')
            continue
        except TypeError:
            pass

        newDic[str(key)] = obj[key]

    return newDic

x = {'Date': datetime.datetime.utcnow(), 34: 89.9, 12.3: 90, 45: 67, 'Extra': 6}

print x

with open('my_dict.json', 'w') as fp:
    json.dump(x, fp, default=outputJSON)

with open('my_dict.json') as f:
    my_dict = json.load(f, object_hook=inputJSON)

print my_dict

Output

{'Date': datetime.datetime(2013, 11, 8, 2, 30, 56, 479727), 34: 89.9, 45: 67, 12.3: 90, 'Extra': 6}
{'Date': datetime.datetime(2013, 11, 8, 2, 30, 56, 479727), 34: 89.9, 45: 67, 12.3: 90, 'Extra': 6}

JSON File

{"Date": "2013-11-08 02:30:56.479727", "34": 89.9, "45": 67, "12.3": 90, "Extra": 6}

This has enabled me to import and export strings, ints, floats and datetime objects. It shouldn't be to hard to extend for other types.

As per the @jjmontes answer, I have used the following approach. For flask and flask-restful users

# get json string
jsonStr = json.dumps(my_dictionary, indent=1, sort_keys=True, default=str)
# then covert json string to json object
return json.loads(jsonStr)

I usually use orjson. Not only because of its tremendous performance, but also for its great (RFC-3339 compliant) support of datetime:

import orjson # via pip3 install orjson
from datetime import datetime

data = {"created_at": datetime(2022, 3, 1)}

orjson.dumps(data) # returns b'{"created_at":"2022-03-01T00:00:00"}'

If you would like to use datetime.datetime objects without a tzinfo as UTC you can add the related option:

orjson.dumps(data, option=orjson.OPT_NAIVE_UTC) # returns b'{"created_at":"2022-03-01T00:00:00+00:00"}'

If you are on both sides of the communication you can use repr() and eval() functions along with json.

import datetime, json

dt = datetime.datetime.now()
print("This is now: {}".format(dt))

dt1 = json.dumps(repr(dt))
print("This is serialised: {}".format(dt1))

dt2 = json.loads(dt1)
print("This is loaded back from json: {}".format(dt2))

dt3 = eval(dt2)
print("This is the same object as we started: {}".format(dt3))

print("Check if they are equal: {}".format(dt == dt3))

You shouldn't import datetime as

from datetime import datetime

since eval will complain. Or you can pass datetime as a parameter to eval. In any case this should work.

If you are working with django models you can directly pass encoder=DjangoJSONEncoder to the field constructor. It will work like a charm.

from django.core.serializers.json import DjangoJSONEncoder 
from django.db import models 
from django.utils.timezone import now


class Activity(models.Model):
    diff = models.JSONField(null=True, blank=True, encoder=DjangoJSONEncoder)


diff = {
    "a": 1,
    "b": "BB",
    "c": now()
}

Activity.objects.create(diff=diff)

I faced this issue today, I found something called pickle. It's a built-in library for serializing python objects and also load it from a pickle file.

The only difference I found between pickle and json is pickle file is a binary file, where as json is a usual text file.

And It doesn't cause any issues with datetime objects.

Another approach is to adopt a concept from FEEL (the Friendly Enough Expression Language) defined in DMN (Decision Model Notation) - namely @strings. Any string starting with @" and ending with " is decoded separately with FEEL decoding. Of course the sender and the receiver have to agree to this convention, but ... the code below lets you encode lots of other things as well as dates, times, date/times, timedeltas. You can encode year/month durations and ranges (so long as you except a 4 element tuple of chr, expr, expr, chr as being a good representation of a range - where the two chrs are open/close brackets). So, @"P4Y2M" is a duration of 4 years and 2 months. @"P2DT5H" is a timedelta of 2 days and 4 hours, @"(2021-01-02 .. 2021-12-31)" is a year range.

The following code can be used to serialize and de-serialize @strings.

import datetime
import pySFeel

parser = pySFeel.SFeelParser()


def convertAtString(thisString):
    # Convert an @string
    (status, newValue) = parser.sFeelParse(thisString[2:-1])
    if 'errors' in status:
        return thisString
    else:
        return newValue


def convertIn(newValue):
    if isinstance(newValue, dict):
        for key in newValue:
            if isinstance(newValue[key], int):
                newValue[key] = float(newValue[key])
            elif isinstance(newValue[key], str) and (newValue[key][0:2] == '@"') and (newValue[key][-1] == '"'):
                newValue[key] = convertAtString(newValue[key])
            elif isinstance(newValue[key], dict) or isinstance(newValue[key], list):
                newValue[key] = convertIn(newValue[key])
    elif isinstance(newValue, list):
        for i in range(len(newValue)):
            if isinstance(newValue[i], int):
                newValue[i] = float(newValue[i])
            elif isinstance(newValue[i], str) and (newValue[i][0:2] == '@"') and (newValue[i][-1] == '"'):
                newValue[i] = convertAtString(newValue[i])
            elif isinstance(newValue[i], dict) or isinstance(newValue[i], list):
                newValue[i] = convertIn(newValue[i])
    elif isinstance(newValue, str) and (newValue[0:2] == '@"') and (newValue[-1] == '"'):
        newValue = convertAtString(newValue)
    return newValue


  def convertOut(thisValue):
      if isinstance(thisValue, datetime.date):
          return '@"' + thisValue.isoformat() + '"'
      elif isinstance(thisValue, datetime.datetime):
          return '@"' + thisValue.isoformat(sep='T') + '"'
      elif isinstance(thisValue, datetime.time):
          return '@"' + thisValue.isoformat() + '"'
      elif isinstance(thisValue, datetime.timedelta):
          sign = ''
          duration = thisValue.total_seconds()
          if duration < 0:
              duration = -duration
              sign = '-'
          secs = duration % 60
          duration = int(duration / 60)
          mins = duration % 60
          duration = int(duration / 60)
          hours = duration % 24
          days = int(duration / 24)
          return '@"%sP%dDT%dH%dM%fS"' % (sign, days, hours, mins, secs)
      elif isinstance(thisValue, bool):
          return thisValue:
      elif thisValue is None:
          return thisValue:
      elif isinstance(thisValue, int):
          sign = ''
          if thisValue < 0:
              thisValue = -thisValue
              sign = '-'
          years = int(thisValue / 12)
          months = (thisValue % 12)
          return '@"%sP%dY%dM"' % (sign, years, months)
      elif isinstance(thisValue, tuple) and (len(thisValue) == 4):
          (lowEnd, lowVal, highVal, highEnd) = thisValue
          return '@"' + lowEnd + str(lowVal) + ' .. ' + str(highVal) + highEnd
      elif thisValue is None:
          return 'null'
      elif isinstance(thisValue, dict):
          for item in thisValue:
              thisValue[item] = convertOut(thisValue[item])
          return thisValue
      elif isinstance(thisValue, list):
          for i in range(len(thisValue)):
              thisValue[i] = convertOut(thisValue[i])
          return thisValue
      else:
          return thisValue