Format floats with standard json module

user_23657 0
Viewed 96021

I am using the standard json module in python 2.6 to serialize a list of floats. However, I'm getting results like this:

>>> import json
>>> json.dumps([23.67, 23.97, 23.87])
'[23.670000000000002, 23.969999999999999, 23.870000000000001]'

I want the floats to be formated with only two decimal digits. The output should look like this:

>>> json.dumps([23.67, 23.97, 23.87])
'[23.67, 23.97, 23.87]'

I have tried defining my own JSON Encoder class:

class MyEncoder(json.JSONEncoder):
    def encode(self, obj):
        if isinstance(obj, float):
            return format(obj, '.2f')
        return json.JSONEncoder.encode(self, obj)

This works for a sole float object:

>>> json.dumps(23.67, cls=MyEncoder)
'23.67'

But fails for nested objects:

>>> json.dumps([23.67, 23.97, 23.87])
'[23.670000000000002, 23.969999999999999, 23.870000000000001]'

I don't want to have external dependencies, so I prefer to stick with the standard json module.

How can I achieve this?

floating-point formatting python json
17 Answers

Here's a solution that worked for me in Python 3 and does not require monkey patching:

import json

def round_floats(o):
    if isinstance(o, float): return round(o, 2)
    if isinstance(o, dict): return {k: round_floats(v) for k, v in o.items()}
    if isinstance(o, (list, tuple)): return [round_floats(x) for x in o]
    return o


json.dumps(round_floats([23.63437, 23.93437, 23.842347]))

Output is:

[23.63, 23.93, 23.84]

It copies the data but with rounded floats.

user_3642480

Using numpy

If you actually have really long floats you can round them up/down correctly with numpy:

import json 

import numpy as np

data = np.array([23.671234, 23.97432, 23.870123])

json.dumps(np.around(data, decimals=2).tolist())

'[23.67, 23.97, 23.87]'

user_4253800

I just released fjson, a small Python library to fix this issue. Install with

pip install fjson

and use just like json, with the addition of the float_format parameter:

import math
import fjson


data = {"a": 1, "b": math.pi}
print(fjson.dumps(data, float_format=".6e", indent=2))

{
  "a": 1,
  "b": 3.141593e+00
}
user_3533370

New Answer:

Inspired by this answer, it looks scary but actually works perfectly

import json

class RoundingFloat(float):
    __repr__ = staticmethod(lambda x: format(x, '.2f'))

json.encoder.c_make_encoder = None

json.encoder.float = RoundingFloat

print(json.dumps({'number': 1.0 / 81}))

Old Answer Below:

I am amazed / bemused that this is not a feature, fortunately, TensorFlow authors have already solved this problem by using regex:

import json
import re

def FormatFloat(json_str, float_digits):
  pattern = re.compile(r'\d+\.\d+')
  float_repr = '{:.' + '{}'.format(float_digits) + 'f}'

  def MRound(match):
    return float_repr.format(float(match.group()))

  return re.sub(pattern, MRound, json_str)

def Dumps(obj, float_digits=-1, **params):
  """Wrapper of json.dumps that allows specifying the float precision used.

  Args:
    obj: The object to dump.
    float_digits: The number of digits of precision when writing floats out.
    **params: Additional parameters to pass to json.dumps.

  Returns:
    output: JSON string representation of obj.
  """
  json_str = json.dumps(obj, **params)

  if float_digits > -1:
    json_str = FormatFloat(json_str, float_digits)

  return json_str

This works by just wrapping json.dumps from the standard package then running a regex on the result.

user_51252640

To achieve the fixed precision float output in the .json file, one way is to make changes in the encoder.py module in python_dir\lib\json module.

I first created a class:

        class FloatRepr(reprlib.Repr):
            def repr_float(self,value,level):
                return format(value,'.2f')

Then, modify the floatstr function to the following:

        def floatstr(o, allow_nan=self.allow_nan, _repr=float.__repr__, _inf=INFINITY,_neginf=-INFINITY):

        if o != o:
            text = 'NaN'
        elif o == _inf:
            text = 'Infinity'
        elif o == _neginf:
            text = '-Infinity'
        else:
            # return _repr(o) # commented out
            return FloatRepr().repr(o) # changes made

        if not allow_nan:
            raise ValueError(
                "Out of range float values are not JSON compliant: " +
                repr(o))

        return text

By doing this, the .json float values will not be a string.

user_198432610

Pros:

  • Works with any JSON encoder, or even python's repr.
  • Short(ish), seems to work.

Cons:

  • Ugly regexp hack, barely tested.

  • Quadratic complexity.

    def fix_floats(json, decimals=2, quote='"'):
    pattern = r'^((?:(?:"(?:\\.|[^\\"])*?")|[^"])*?)(-?\d+\.\d{'+str(decimals)+'}\d+)'
    pattern = re.sub('"', quote, pattern) 
    fmt = "%%.%df" % decimals
    n = 1
    while n:
        json, n = re.subn(pattern, lambda m: m.group(1)+(fmt % float(m.group(2)).rstrip('0')), json)
    return json
user_2182940

I did that :) Beware that with my code you will always have 2 digit's after comma

>>> json_dumps_with_two_digit_float({'a': 1.0})
'{"a": 1.00}'

My custom function:

from unittest.mock import patch
import json
# We need to ensure that c encoder will not be launched
@patch('json.encoder.c_make_encoder', None)
def json_dumps_with_two_digit_float(some_object):
    # saving original method
    of = json.encoder._make_iterencode
    def inner(*args, **kwargs):
        args = list(args)
        # fifth argument is float formater which will we replace
        args[4] = lambda o: '{:.2f}'.format(o)
        return of(*args, **kwargs)
    
    with patch('json.encoder._make_iterencode', wraps=inner):
        return json.dumps(some_object)

Don't forget to create some tests in your project, because my func heavily related to python json module implementation which can be changed in the future.

user_58058360
Related