json.dump not calling default or cls

user_3535456 0
Viewed 4153

Trying to serialize a dict containing datetime objects as keys to json. A few other answers have suggested using the json.dump parameters default or cls however, neither of these methods appear to be even called. See MWEs below. What am I missing?

Using Default

from datetime import datetime
import json

def default(obj):
    print("Default Called")
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()

test = {datetime(1970, 1, 1, 8, 0, 0): 10}

with open("output.json", "w") as fo:
    json.dump(test, fo, default=default)

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    json.dump(test, fo, default=default)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 376, in _iterencode_dict
    raise TypeError(f'keys must be str, int, float, bool or None, '
TypeError: keys must be str, int, float, bool or None, not datetime

Using cls

from datetime import datetime
import json

class DateEncoder(json.JSONEncoder):
    def default(self, obj):
        print("Default called")
        if isinstance(obj, (datetime, date)):
            return obj.isoformat()

        # Let the base class default method raise the TypeError
        return json.JSONEncoder.default(self, obj)

test = {datetime(1970, 1, 1, 8, 0, 0): 10}

with open("output.json", "w") as fo:
    json.dump(test, fo, cls=DateEncoder)

Traceback (most recent call last):
  File "test.py", line 16, in <module>
    json.dump(test, fo, cls=DateEncoder)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 376, in _iterencode_dict
    raise TypeError(f'keys must be str, int, float, bool or None, '
TypeError: keys must be str, int, float, bool or None, not datetime
python-3.x datetime python json
2 Answers

TLDR: .default is only called for dict values, not keys.

To elaborate on Abhishek Kumars answer; after reviewing the json.JSONEncoder source, .default is only ever called on dict values that are not serializable, never keys. Keys must be one of str, int, float, bool or None.

To use datetime objects as keys, .iterencode must be overwritten to use recursive pre-processor method to convert datetime to str:

test = {
        "key_1": "Value_1",
        "key_2": 10,
        "key_3": ["list_" + str(i) for i in range(5)],
        "key_4": {"nestkey_" + str(i) : "nestvalue_" + str(i) for i in range(5) },
        "key_5": datetime.datetime(1970, 1, 1, 8, 0, 0),
        datetime.datetime(1970, 1, 1, 8, 0, 0): "datetime_key"
}

class DateTimeEncoder(json.JSONEncoder):

    def _preprocess_date(self, obj):
        if isinstance(obj, (datetime.date, datetime.datetime, datetime.timedelta)):
            return str(obj)
        elif isinstance(obj, dict):
            return {self._preprocess_date(k): self._preprocess_date(v) for k,v in obj.items()}
        elif isinstance(obj, list):
            return [self._preprocess_date(i) for i in obj]
        return obj

    def default(self, obj):
        if isinstance(obj, (datetime.date, datetime.datetime, datetime.timedelta)):
            return str(obj)
        return super().default(obj)

    def iterencode(self, obj):
        return super().iterencode(self._preprocess_date(obj))

with open('output.json', 'w') as fo:
    json.dump(test, fo, cls=DateTimeEncoder)

This is obviously an expensive operation for larger dictionaries so some caution should be exercised. Also, would be nice for json.JSONEncoder to be updated to use default on keys as well as values - issue created at : https://bugs.python.org/issue41569

user_35354560

you can try this :

class DatesToStrings(json.JSONEncoder):
    def _encode(self, obj):
        if isinstance(obj, dict):
            def transform_date(o):
                return self._encode(o.isoformat() if isinstance(o, datetime) else o)
            return {transform_date(k): transform_date(v) for k, v in obj.items()}
        else:
            return obj

    def encode(self, obj):
        return super(DatesToStrings, self).encode(self._encode(obj))

with open("output.json", "w") as fo:
    json.dump( json.dumps( test, cls=DatesToStrings), fo, cls=DatesToStrings)
user_74427840
Related