How to flatten a nested dict and use inner values on collision

user_4981251 0
Viewed 326

Question

For the following dictionary:

{'id': 1, 'label': 'hello', 'remove_me': {'world': {'keep_me': 52}}}

I want to create a new dictionary without the remove_me or world keys.

Edit update:

To summarise, I wanted to do the following:

If the values of an item are a nested dictionary. Update the new dictionary with the inner result while removing the current key value from the main dict.

If the values of an item are not a nested dictionary, update the new dictionary with the key, value.

The accepted answer covers this.

What have I tried?

{k:v for (k,v) in d.items() if not k == 'remove_me'}

yields:

{'id': 1, 'label': 'hello'}

not exactly what I need as I'm dropping the nested dict.

Desired output:

{'id': 1, 'label': 'hello','keep_me': 52}
flatten dictionary python
5 Answers
dico = {'id': 1, 'label': 'hello', 'remove_me': {'world': {'keep_me': 52}}}

# Just what you have done
new_dico = {k:v for (k,v) in dico.items() if not k == 'remove_me'}

# Plus this line
new_dico.update(dico['remove_me']['world'])

print(new_dico)
# {'id': 1, 'label': 'hello', 'keep_me': 52}

Inspired by what I have read here, a flatten function for the main dict, whatever deep one of your key-dict is:

dico = {'id': 1, 'label': 'hello', 'remove_me': {'world': {'keep_me': 52}}}

def dFlatten(dico, d = {}):
    for k, v in dico.items():
        if isinstance(v, dict):
            dFlatten(v)
        else:
            d[k] = v
    return d
dico = dFlatten(dico)
print(dico)
# {'id': 1, 'label': 'hello', 'keep_me': 52}

For instance with a deeper dico :

dico2 = {'id': 1, 'label': 'hello', 'stuff1': {'stuff2': {'remove_me': {'world': {'keep_me': 52}}}}}
dico2 = dFlatten(dico2)
print(dico2)         
# {'id': 1, 'label': 'hello', 'keep_me': 52}

With several deep keys with the same dFlatten function

dico3 = {'id': 1, 'label': 'hello', 'deep': {'L1': {'L2': 52}}, 'remove_me': {'world': {'keep_me': 52}}}
dico3 = dFlatten(dico3)
print(dico3)         
# {'id': 1, 'label': 'hello', 'keep_me': 52, 'L2': 52}
user_134605430

You might need to be a bit more specific about the structure of your dictionary, and how to handle multiple keys. Regardless, here's one recursive method that fits your description and attempts to preserve as many keys as possible.

def clean_kvp(k, v, invalid_keys=["remove_me", "world"]):
    if k not in invalid_keys:
        return [(k, v)]
    if not isinstance(v, dict):
        return []
    return [
        (kkk, vvv)
        for kk, vv in v.items()
        for kkk, vvv in clean_kvp(kk, vv)
    ]

def clean_dict(d):
    return {
        kk: vv
        for k, v in d.items()
        for kk, vv in clean_kvp(k, v)
    }

A couple of tests:

>>> d = {'id': 1, 'label': 'hello', 'remove_me': {'world': {'keep_me': 52}}}
>>> clean_dict(d)
{'id': 1, 'label': 'hello', 'keep_me': 52}

>>> d = {
...     'id': 1,
...     'label': 'hello',
...     'remove_me': {'world': {'keep_me': 52, 'test': 2}}
... }
>>> clean_dict(d)
{'id': 1, 'label': 'hello', 'keep_me': 52, 'test': 2}

>>> d = {'id': 1, 'label': {'test': 'hello'}}
>>> clean_dict(d)
{'id': 1, 'label': {'test': 'hello'}}
user_3651020

You can try

d = {'id': 1, 'label': 'hello', 'remove_me': {'world': {'keep_me': 52}}}
for k, v in list(d.items()):
    if isinstance(v, dict):
        for i in v:
           if isinstance(v[i], dict):
               d.update(v[i])
           else:
               d.update(v)
        del d[k]
print(d)

Output

{'id': 1, 'label': 'hello', 'keep_me': 52}

This code will check if the values of each item are a dict and if so it will update the dict with the inner result and remove the current key value from the main dict. By that, in the end, the d dict will stay only with a key and a string value without nested dicts.

user_131012390

Flattening a dictionary is a whole thing in itself. Does this accomplish your goal? (Untested, I'm typing on my phone a.t.m.):

def flattenDict(myDict, blacklist):
  returnDict = {}
  for key, val in myDict:
    If isinstance(val, "dict"):
       myDict.update(flattenDict(val))
    elif key in blacklist:
       continue 
    elif val in blacklist:
        continue
    returnDict[key] = val
  return returnDict 

cleanDict = flattenDict(myDict, ["remove_me", "world"])
user_97733510

I think this can be done in a general case with a single recursive function with something like:

def remove_keys(d, keys):
    if not isinstance(d, dict):
        return d 
    ret = {}
    for k, v in d.items():
        clean = remove_keys(v, keys)
        if k not in bad_keys:
            ret[k] = clean
        else:
            if isinstance(clean, dict):
                ret.update(clean)

    return ret

bad_keys = ['remove_me', 'world']

d = {'id': 1, 'label': 'hello', 'remove_me': {'world': {'keep_me': 52}}}
remove_keys(d, bad_keys)
# {'id': 1, 'label': 'hello', 'keep_me': 52}

d = {'id': 1, 'label': 'hello', 'remove_me': 52}
remove_keys(d, bad_keys)
# {'id': 1, 'label': 'hello'}

d = {'id': 1, 'label': 'hello', 'dont_remove_me': {'world': {'keep_me': 52}}}
remove_keys(d, bad_keys)
# {'id': 1, 'label': 'hello', 'dont_remove_me': {'keep_me': 52}}
user_38746230
Related