Simplify a deep dictionary in Python

user_7273971 0
Viewed 160

I have a rather deep dict that I need to simplify. And I've encountered some problems by doing that.

Here is a small sample of the dictionary that needs to be simplified:

data_dict = {
    "DATA": {
        "Page1": [{
            "Section": [{
                "Name": [{
                    "text": "John"
                }],
                "ID_Number": [{
                    "text": "123456"
                }]
            }]
        }],
        "Page2": [{
            "Section": [{
                "Name": [{
                    "text": "Rob"
                }],
                "ID_Number": [{
                    "text": "654321"
                }]
            }]
        }]
    }
}

What I've done already:

my_dict = {}
for value in data_dict.values():
    for key, val in value.items():
        if "Tab" in key:
            my_dict[key] = val
        if type(val) == list:
            for i in val:
                for key1, val1 in i.items():
                    my_dict[key] = val1


result_dict = {}
page_list = []
for keys, values in my_dict.items():
    for val in values:
        if type(val) != str:
            for key1, val1 in val.items():
                for x in val1:
                    result_dict[key1] = x.get('text')
                    page_list.append(result_dict)
                    my_dict[keys] = page_list
print("my_dict = ", my_dict)

Current result:

my_dict = {'Page1': [{'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '65432
1'}, {'Name': 'Rob', 'ID_Number': '654321'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '
654321'}, {'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '654321'}]}

The problem is that result_dict is being appended to page_list more than once which is unnecessary. Also, my approach is very messy. Is there a cleaner way to get the same result?

Desired result:

my_dict = {"Page1": [{"Name": "John", "ID_Number": "123456"}], "Page2": [{"Name": "Rob", "ID_Number": "654321"}]}
data-manipulation dictionary python
2 Answers

Solution 1 (less loops, but added if statements):

If you want to avoid too many nested for loops. I would take advantage of knowing before-hand the duplicate keys and use that information to easily get to the inner keys or values.

Reference to dict for solution 1 & 2:

data_dict = {"DATA": {"Page1": [{"Section": [{"Name": [{"text": "John"}],"ID_Number": [{"text": "123456"}]}]}],"Page2": [{"Section": [{"Name": [{"text": "Rob"}],"ID_Number": [{"text": "654321"}]}]}]}}

Code:

# Depth #1
old_dict = data_dict["DATA"]
new_dict = {}

for d1_key in old_dict:
    d2 = old_dict[d1_key][0]["Section"][0]
    for d2_key in d2:
        if d2_key == "Name":
            new_dict[d1_key] = [{d2_key: d2[d2_key][0]["text"]}]
        if d2_key == "ID_Number":
            merge = new_dict[d1_key][0]
            # Merge above if statement (dict merging)
            new_dict[d1_key] = [{**merge, **{d2_key:d2[d2_key][0]["text"]}}]
print(new_dict)

Output:

{'Page1': [{'Name': 'John', 'ID_Number': '123456'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}]}

Solution 2: (more for loops, more readible)

(Recommended) Here is a second solution that gives the same desired output that does not take advantage of information about the keys or values, but only looks at the structure of the data. I prefer this one as it is easy to read, modify or extend!

Code:

# Depth #1
old_dict = data_dict["DATA"]
new_dict = {}

unlist = 0
k3_temp = None # instead of merge
v4_temp = None
for k1, v1 in old_dict.items():
    for v2 in v1[unlist].values(): # using values because we don't use the Section key
        for k3, v3 in v2[unlist].items():
            for k4, v4 in v3[unlist].items():
                new_dict[k1] = [{k3_temp:v4_temp, k3:v4}]
                k3_temp = k3
                v4_temp = v4

print(new_dict)

Output:

{'Page1': [{'Name': 'John', 'ID_Number': '123456'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}]}
user_24607720

Just to see another solution with a ridiculous amount of for loops:

new_dic = {}
inner_list = []

for i in data_dict:
    for j in data_dict[i]:
        for k in data_dict[i][j]:
            for m in k:
                for n in k[m]:
                    for x in n:
                        for y in n[x]:
                            for keys, values in y.items():
                                inner_list.append(values)
                    new_dic[j] = [{'Name': inner_list[0], 'ID_Number': inner_list[1]}]
                    inner_list = [] 
                                    
print(new_dic)

output

{'Page1': [{'Name': 'John', 'ID_Number': '123456'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}]}
user_93071410
Related