How to merge nested dictionary having same keys in python

Viewed 112

I have a data structure like this:

[ {'SNAPSHOT': {'SnapshotVersion': '304'}},

  {'SNAPSHOT': {'SnapshotCreationDate': '2015-06-21 17:33:41'}},


  {'CafeData': {'CafeVersion': '2807'}}, 

  {'CafeData': {'IsSoftwareOnly': '1'}}, 

  {'CafeData'{'IsPassportTCPIP': '1'}} 

  {'SNAPSHOT': {'SnapshotVersion': '777'}},

  {'SNAPSHOT': {'SnapshotCreationDate': '2017-07-27 17:37:47'}},]

The output should like this:

 [ {'SNAPSHOT': {'SnapshotVersion': '304','SnapshotCreationDate': '2015-06-21 17:33:41'}},

   {'CafeData': {'CafeVersion': '2807','IsSoftwareOnly': '1','IsPassportTCPIP': '1'}} 
 
   {'SNAPSHOT': {'SnapshotVersion': '777','SnapshotCreationDate': '2017-07-27 17:37:47'}},
]

or the output should like this:

[ {'SNAPSHOT': {'SnapshotVersion': ['304','777'],
                'SnapshotCreationDate': ['2015-06-21 17:33:41','2017-07-27 17:37:47']}},

   {'CafeData': {'CafeVersion': '2807','IsSoftwareOnly': '1','IsPassportTCPIP': '1'}} 

]
2 Answers

Could use a for loop and keep track of the last key that was merged.

from pprint import pprint

data = [
    {"SNAPSHOT": {"SnapshotVersion": "304"}},
    {"SNAPSHOT": {"SnapshotCreationDate": "2015-06-21 17:33:41"}},
    {"CafeData": {"CafeVersion": "2807"}},
    {"CafeData": {"IsSoftwareOnly": "1"}},
    {"CafeData": {"IsPassportTCPIP": "1"}},
    {"SNAPSHOT": {"SnapshotVersion": "777"}},
    {"SNAPSHOT": {"SnapshotCreationDate": "2017-07-27 17:37:47"}},
]

last_key = None
grouped = []
for value in data:
    # Easy way to get the key of a dict with one key
    curr_key = next(iter(value))
    # Decide if we should work on the next entry
    if last_key is None or curr_key != last_key:
        grouped.append(value)
    else:
        # update the last value in the group with the new data
        grouped[-1][curr_key].update(value[curr_key])
    # Move to the next item
    last_key = curr_key

pprint(grouped)
[{'SNAPSHOT': {'SnapshotCreationDate': '2015-06-21 17:33:41',
               'SnapshotVersion': '304'}},
 {'CafeData': {'CafeVersion': '2807',
               'IsPassportTCPIP': '1',
               'IsSoftwareOnly': '1'}},
 {'SNAPSHOT': {'SnapshotCreationDate': '2017-07-27 17:37:47',
               'SnapshotVersion': '777'}}]

Option 2 is similar, but for grouped I would just use a dict. In this case you don't need to know about the last key.

Also you need to make some assumptions on how to merge vales if there are key collisions in the nested dict.

grouped = {}
for value in data:
    curr_key = next(iter(value))
    curr_value = value[curr_key]
    group = grouped.setdefault(curr_key, {})

    for sub_key, sub_value in curr_value.items():
        # check if you need to merge
        if sub_key in group:
            # If the key is already present, but is not a list, make it one
            if not isinstance(group[sub_key], list):
                group[sub_key] = [group[sub_key]]
            # Add the new value to the list
            group[sub_key].append(sub_value)
        else:
            # Otherwise just copy it over
            group[sub_key] = sub_value

pprint(grouped)
{'CafeData': {'CafeVersion': '2807',
              'IsPassportTCPIP': '1',
              'IsSoftwareOnly': '1'},
 'SNAPSHOT': {'SnapshotCreationDate': ['2015-06-21 17:33:41',
                                       '2017-07-27 17:37:47'],
              'SnapshotVersion': ['304', '777']}}

Here's a variant that uses [Python.Docs]: itertools.groupby(iterable, key=None).

code00.py:

#!/usr/bin/env python

import sys
import itertools as it
from pprint import pprint as pp


l = [
    {"SNAPSHOT": {"SnapshotVersion": "304"}},
    {"SNAPSHOT": {"SnapshotCreationDate": "2015-06-21 17:33:41"}},
    {"CafeData": {"CafeVersion": "2807"}},
    {"CafeData": {"IsSoftwareOnly": "1"}},
    {"CafeData": {"IsPassportTCPIP": "1"}},
    {"SNAPSHOT": {"SnapshotVersion": "777"}},
    {"SNAPSHOT": {"SnapshotCreationDate": "2017-07-27 17:37:47"}},
]

def main(*argv):
    res = []
    for k, g in it.groupby(l, key=lambda x: next(iter(x))):
        res_item = {}
        for item in g:
            res_item.update(item[k])
        res.append({k: res_item})
    pp(res)


if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.")
    sys.exit(rc)

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q067328076]> "e:\Work\Dev\VEnvs\py_pc064_03.08.07_test0\Scripts\python.exe" code00.py
Python 3.8.7 (tags/v3.8.7:6503f05, Dec 21 2020, 17:59:51) [MSC v.1928 64 bit (AMD64)] 64bit on win32

[{'SNAPSHOT': {'SnapshotCreationDate': '2015-06-21 17:33:41',
               'SnapshotVersion': '304'}},
 {'CafeData': {'CafeVersion': '2807',
               'IsPassportTCPIP': '1',
               'IsSoftwareOnly': '1'}},
 {'SNAPSHOT': {'SnapshotCreationDate': '2017-07-27 17:37:47',
               'SnapshotVersion': '777'}}]

Done.
Related