Python JSON serialize a Decimal object

Viewed 322473

I have a Decimal('3.9') as part of an object, and wish to encode this to a JSON string which should look like {'x': 3.9}. I don't care about precision on the client side, so a float is fine.

Is there a good way to serialize this? JSONDecoder doesn't accept Decimal objects, and converting to a float beforehand yields {'x': 3.8999999999999999} which is wrong, and will be a big waste of bandwidth.

19 Answers

How about subclassing json.JSONEncoder?

class DecimalEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, decimal.Decimal):
            # wanted a simple yield str(o) in the next line,
            # but that would mean a yield on the line with super(...),
            # which wouldn't work (see my comment below), so...
            return (str(o) for o in [o])
        return super(DecimalEncoder, self).default(o)

Then use it like so:

json.dumps({'x': decimal.Decimal('5.5')}, cls=DecimalEncoder)

The native Django option is missing so I'll add it for the next guy/gall that looks for it.

Starting on Django 1.7.x there is a built-in DjangoJSONEncoder that you can get it from django.core.serializers.json.

import json
from django.core.serializers.json import DjangoJSONEncoder
from django.forms.models import model_to_dict

model_instance = YourModel.object.first()
model_dict = model_to_dict(model_instance)

json.dumps(model_dict, cls=DjangoJSONEncoder)

Presto!

For Django users:

Recently came across TypeError: Decimal('2337.00') is not JSON serializable while JSON encoding i.e. json.dumps(data)

Solution:

# converts Decimal, Datetime, UUIDs to str for Encoding
from django.core.serializers.json import DjangoJSONEncoder  

json.dumps(response.data, cls=DjangoJSONEncoder)

But, now the Decimal value will be a string, now we can explicitly set the decimal/float value parser when decoding data, using parse_float option in json.loads:

import decimal 

data = json.loads(data, parse_float=decimal.Decimal) # default is float(num_str)

3.9 can not be exactly represented in IEEE floats, it will always come as 3.8999999999999999, e.g. try print repr(3.9), you can read more about it here:

http://en.wikipedia.org/wiki/Floating_point
http://docs.sun.com/source/806-3568/ncg_goldberg.html

So if you don't want float, only option you have to send it as string, and to allow automatic conversion of decimal objects to JSON, do something like this:

import decimal
from django.utils import simplejson

def json_encode_decimal(obj):
    if isinstance(obj, decimal.Decimal):
        return str(obj)
    raise TypeError(repr(obj) + " is not JSON serializable")

d = decimal.Decimal('3.5')
print simplejson.dumps([d], default=json_encode_decimal)

For those who don't want to use a third-party library... An issue with Elias Zamaria's answer is that it converts to float, which can run into problems. For example:

>>> json.dumps({'x': Decimal('0.0000001')}, cls=DecimalEncoder)
'{"x": 1e-07}'
>>> json.dumps({'x': Decimal('100000000000.01734')}, cls=DecimalEncoder)
'{"x": 100000000000.01733}'

The JSONEncoder.encode() method lets you return the literal json content, unlike JSONEncoder.default(), which has you return a json compatible type (like float) that then gets encoded in the normal way. The problem with encode() is that it (normally) only works at the top level. But it's still usable, with a little extra work (python 3.x):

import json
from collections.abc import Mapping, Iterable
from decimal import Decimal

class DecimalEncoder(json.JSONEncoder):
    def encode(self, obj):
        if isinstance(obj, Mapping):
            return '{' + ', '.join(f'{self.encode(k)}: {self.encode(v)}' for (k, v) in obj.items()) + '}'
        if isinstance(obj, Iterable) and (not isinstance(obj, str)):
            return '[' + ', '.join(map(self.encode, obj)) + ']'
        if isinstance(obj, Decimal):
            return f'{obj.normalize():f}'  # using normalize() gets rid of trailing 0s, using ':f' prevents scientific notation
        return super().encode(obj)

Which gives you:

>>> json.dumps({'x': Decimal('0.0000001')}, cls=DecimalEncoder)
'{"x": 0.0000001}'
>>> json.dumps({'x': Decimal('100000000000.01734')}, cls=DecimalEncoder)
'{"x": 100000000000.01734}'

You can create a custom JSON encoder as per your requirement.

import json
from datetime import datetime, date
from time import time, struct_time, mktime
import decimal

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, datetime):
            return str(o)
        if isinstance(o, date):
            return str(o)
        if isinstance(o, decimal.Decimal):
            return float(o)
        if isinstance(o, struct_time):
            return datetime.fromtimestamp(mktime(o))
        # Any other serializer if needed
        return super(CustomJSONEncoder, self).default(o)

The Decoder can be called like this,

import json
from decimal import Decimal
json.dumps({'x': Decimal('3.9')}, cls=CustomJSONEncoder)

and the output will be:

>>'{"x": 3.9}'

Based on stdOrgnlDave answer I have defined this wrapper that it can be called with optional kinds so the encoder will work only for certain kinds inside your projects. I believe the work should be done inside your code and not to use this "default" encoder since "it is better explicit than implicit", but I understand using this will save some of your time. :-)

import time
import json
import decimal
from uuid import UUID
from datetime import datetime

def JSONEncoder_newdefault(kind=['uuid', 'datetime', 'time', 'decimal']):
    '''
    JSON Encoder newdfeault is a wrapper capable of encoding several kinds
    Use it anywhere on your code to make the full system to work with this defaults:
        JSONEncoder_newdefault()  # for everything
        JSONEncoder_newdefault(['decimal'])  # only for Decimal
    '''
    JSONEncoder_olddefault = json.JSONEncoder.default

    def JSONEncoder_wrapped(self, o):
        '''
        json.JSONEncoder.default = JSONEncoder_newdefault
        '''
        if ('uuid' in kind) and isinstance(o, uuid.UUID):
            return str(o)
        if ('datetime' in kind) and isinstance(o, datetime):
            return str(o)
        if ('time' in kind) and isinstance(o, time.struct_time):
            return datetime.fromtimestamp(time.mktime(o))
        if ('decimal' in kind) and isinstance(o, decimal.Decimal):
            return str(o)
        return JSONEncoder_olddefault(self, o)
    json.JSONEncoder.default = JSONEncoder_wrapped

# Example
if __name__ == '__main__':
    JSONEncoder_newdefault()

If someone is still looking for the answer, it is most probably you have a 'NaN' in your data that you are trying to encode. Because NaN is considered as float by Python.

For anybody that wants a quick solution here is how I removed Decimal from my queries in Django

total_development_cost_var = process_assumption_objects.values('total_development_cost').aggregate(sum_dev = Sum('total_development_cost', output_field=FloatField()))
total_development_cost_var = list(total_development_cost_var.values())
  • Step 1: use , output_field=FloatField() in you r query
  • Step 2: use list eg list(total_development_cost_var.values())

Hope it helps

This question is old, but there seems to be a better and much simpler solution in Python3 for most use-cases:

number = Decimal(0.55)
converted_number = float(number) # Returns: 0.55 (as type float)

You can just convert Decimal to float.

My 2 cents for easy solution, if you're sure Decimal is the only bad guy on your json dumps method:

print(json.loads(json.dumps({
    'a': Decimal(1230),
    'b': Decimal(11111111123.22),
}, default=lambda x: eval(str(x)))))

>>> {'a': 1230, 'b': 11111111123.22}

The "smart" thing here is using default to convert Decimal to int or float, automatically, taking advantage of eval function: default=lambda x: eval(str(x))

But always be careful using eval on your code as it can lead to security issues ;)

Related