Recursive enumerate JSON hierarchy parent/child to dictionary

user_5222018 0
Viewed 370

I'm looking to flatten a JSON hierarchy of unknown structure to a dictionary, capturing the full key hierarchy in the dictionary result to uniquely identify it.

So far I am able to print the key:value pair for the all the parent/child nodes recursively but I am having trouble:

(1) figuring out how to pass the parent hierarchy keys for a recursive (child) execution and then reset it when it exits the child key.

(2) writing to a single dictionary result - when I define the dictionary within the recursive function, I end up creating multiple dictionaries ... Do I need to wrap this function in a master function to avoid this?

Thanks!

# flatten/enumerate example I'm using

with open('_json\\file.json') as f:
    data = json.load(f)

def parse_json_response(content):

    if len (content.keys()) > 1 :
        for key, value in content.items():
            if type(value) is dict:
                parse_json_response(value)
            else:
                print('{}:{}'.format(key,value))
    else:
        print(value)

if __name__ == '__main__':
    parse_json_response(data)

# current result as print
id = 12345
firstName = John
lastName = Smith
DOB = 1980-01-01
phone = 123
line1 = Unit 4
line2 = 3 Main st

# desired result to dictionary {}
id = 12345
fields.firstName = John
fields.lastName = Smith
fields.DOB = 1980-01-01
fields.phone = 123
fields.address.residential.line1 = Unit 4
fields.address.residential.line2 = 3 Main st
recursion enumerate dictionary python json
2 Answers

You can create the flattened dictionary (rather than just print values), by keeping track of the parent and recursing in the correct spot. That might look something like:

d = {
    "ID": "12345",
    "fields": {
        "firstName": "John",
        "lastName": "Smith",
        "DOB": "1980-01-01",
        "phoneLand": "610292659333",
        "address": {
            "residential": {
                "line1": "Unit 4",
                "line2": "3 Main st"
            }
        }
    }
}

def flattenDict(d, parent=None):
    ret = {}
    for k, v in d.items():
        if parent:
            k = f'{parent}.{k}'
        if isinstance(v, dict):
            ret.update(flattenDict(v, k))
        else:
            ret[k] = v
    return ret

flat = flattenDict(d)

flat will be:

{'ID': '12345',
 'fields.firstName': 'John',
 'fields.lastName': 'Smith',
 'fields.DOB': '1980-01-01',
 'fields.phoneLand': '610292659333',
 'fields.address.residential.line1': 'Unit 4',
 'fields.address.residential.line2': '3 Main st'}

You can also arrange the output to be a generator that yields tuples. You can then pass this to dict() for the same result:

def flattenDict(d):
    for k, v in d.items():
        if isinstance(v, dict):
            yield from ((f'{k}.{kk}', v) for kk, v in flattenDict(v))
        else:
            yield (k, v)

dict(flattenDict(d))
user_38746230

Try this below:

test = {
    "ID": "12345",
    "fields": {
        "firstName": "John",
        "lastName": "Smith",
        "DOB": "1980-01-01",
        "phoneLand": "610292659333",
        "address": {
            "residential": {
                "line1": "Unit 4",
                "line2": "3 Main st"
            }
        }
    }
}

def func(d, parent=""):
    for key, value in d.items():
        if isinstance(value, dict):
            func(value, parent=parent+key+".")
        else:
            print(f"{parent+key} = {value}")


func(test)

Result:

ID = 12345
fields.firstName = John
fields.lastName = Smith
fields.DOB = 1980-01-01
fields.phoneLand = 610292659333
fields.address.residential.line1 = Unit 4
fields.address.residential.line2 = 3 Main st
user_118112550
Related