Python json.load JSONDecodeError: Extra data error when trying load multiple json dictionaries

user_11375718 0
Viewed 2724

So I have to import a json file in an identical format shown below:

{
    "name": "bob"
}
{
    "name": "sarah"
}

This is the function I am trying to use to open it:

def read_json_file(file):
    with open(file, "r") as r:
        response = json.load(r)
        return response

I am getting this error when trying to load it:

json.decoder.JSONDecodeError: Extra data: line 4 column 1 (char 22)

There's no way for me to fix the json data as the file is quite large. I need a way to work around it to parse through each dictionary.

I have already tried the method when this question was asked:

Python json.loads shows ValueError: Extra data

I tried changing my function to match the top answer:

    response = json.dumps(r)

That however brought upon this error:

TypeError: Object of type TextIOWrapper is not JSON serializable

Any help would be appreciated on this.

jsondecoder python json
2 Answers

In order solving that kind of "multiple"/"invalid" JSON, you can read the entire file, add these brackets [] to encapsulate the string and then load it as string with json.loads().

  1. Read whole file as string, store it into a variable.
  2. Remove all occurrences of newlines and spaces.
  3. Add the comma , the intersection }{, so it will be ...},{....
  4. Encapsulate it with the brackets [].
  5. Use json.loads() to parse the JSON string.

Full code:

def read_json_file(file):
    with open(file, "r") as r:
        response = r.read()
        response = response.replace('\n', '')
        response = response.replace('}{', '},{')
        response = "[" + response + "]"
        return json.loads(response)
user_146318850

You can use JSONDecoder.raw_decode to incrementally consume the input. Here's an example based on the source of decode():

def json_decode_many(s):
  import json
  import json.decoder
  decoder = json.JSONDecoder()
  _w = json.decoder.WHITESPACE.match

  idx = 0

  while True:
    idx = _w(s, idx).end() # skip leading whitespace
    if idx >= len(s):
      break
    obj, idx = decoder.raw_decode(s, idx=idx)
    yield obj

Then usage looks like

>>> input_string = """
{
  "name": "bob"
}
{
  "name": "sarah"
}
"""
>>> for x in json_decode_many(input_string):
...   print("Decoded:", x)
...
Decoded: {'name': 'bob'}
Decoded: {'name': 'sarah'}
orip_370200
Related