Update value of a nested dictionary of varying depth

If you happen to be using pydantic (great lib, BTW), you can use one of its utility methods:

from pydantic.utils import deep_update

dictionary1 = deep_update(dictionary1, update)

UPDATE: reference to code, as pointed by @Jorgu. If installing pydantic is not desired, the code is short enough to be copied, provided adequate licenses compatibilities.

Just use python-benedict (I did it), it has a merge (deepupdate) utility method and many others. It works with python 2 / python 3 and it is well tested.

from benedict import benedict

dictionary1=benedict({'level1':{'level2':{'levelA':0,'levelB':1}}})
update={'level1':{'level2':{'levelB':10}}}
dictionary1.merge(update)
print(dictionary1)
# >> {'level1':{'level2':{'levelA':0,'levelB':10}}}

Installation: pip install python-benedict

Documentation: https://github.com/fabiocaccamo/python-benedict

Note: I am the author of this project

This question is old, but I landed here when searching for a "deep merge" solution. The answers above inspired what follows. I ended up writing my own because there were bugs in all the versions I tested. The critical point missed was, at some arbitrary depth of the two input dicts, for some key, k, the decision tree when d[k] or u[k] is not a dict was faulty.

Also, this solution does not require recursion, which is more symmetric with how dict.update() works, and returns None.

import collections
def deep_merge(d, u):
   """Do a deep merge of one dict into another.

   This will update d with values in u, but will not delete keys in d
   not found in u at some arbitrary depth of d. That is, u is deeply
   merged into d.

   Args -
     d, u: dicts

   Note: this is destructive to d, but not u.

   Returns: None
   """
   stack = [(d,u)]
   while stack:
      d,u = stack.pop(0)
      for k,v in u.items():
         if not isinstance(v, collections.Mapping):
            # u[k] is not a dict, nothing to merge, so just set it,
            # regardless if d[k] *was* a dict
            d[k] = v

        else:
            # note: u[k] is a dict
            if k not in d:
                # add new key into d
                d[k] = v
            elif not isinstance(d[k], collections.Mapping):
                # d[k] is not a dict, so just set it to u[k],
                # overriding whatever it was
                d[k] = v
            else:
                # both d[k] and u[k] are dicts, push them on the stack
                # to merge
                stack.append((d[k], v))

The code below should solve the update({'k1': 1}, {'k1': {'k2': 2}}) issue in @Alex Martelli's answer the right way.

def deepupdate(original, update):
    """Recursively update a dict.

    Subdict's won't be overwritten but also updated.
    """
    if not isinstance(original, abc.Mapping):
        return update
    for key, value in update.items():
        if isinstance(value, abc.Mapping):
            original[key] = deepupdate(original.get(key, {}), value)
        else:
            original[key] = value
    return original

I recommend to replace {} by type(v)() in order to propagate object type of any dict subclass stored in u but absent from d. For example, this would preserve types such as collections.OrderedDict:

Python 2:

import collections

def update(d, u):
    for k, v in u.iteritems():
        if isinstance(v, collections.Mapping):
            d[k] = update(d.get(k, type(v)()), v)
        else:
            d[k] = v
    return d

Python 3:

import collections.abc

def update(d, u):
    for k, v in u.items():
        if isinstance(v, collections.abc.Mapping):
            d[k] = update(d.get(k, type(v)()), v)
        else:
            d[k] = v
    return d

Thanks to hobs for his comment on Alex's answer. Indeed update({'k1': 1}, {'k1': {'k2': 2}}) will cause TypeError: 'int' object does not support item assignment.

We should check the types of the input values at the beginning of the function. So, I suggest the following function, which should solve this (and other) problem.

Python 3:

from collections.abc import Mapping


def deep_update(d1, d2):
    if all((isinstance(d, Mapping) for d in (d1, d2))):
        for k, v in d2.items():
            d1[k] = deep_update(d1.get(k), v)
        return d1
    return d2

def update(value, nvalue):
    if not isinstance(value, dict) or not isinstance(nvalue, dict):
        return nvalue
    for k, v in nvalue.items():
        value.setdefault(k, dict())
        if isinstance(v, dict):
            v = update(value[k], v)
        value[k] = v
    return value

use dict or collections.Mapping

I know this question is pretty old, but still posting what I do when I have to update a nested dictionary. We can use the fact that dicts are passed by reference in python Assuming that the path of the key is known and is dot separated. Forex if we have a dict named data:

{
"log_config_worker": {
    "version": 1, 
    "root": {
        "handlers": [
            "queue"
        ], 
        "level": "DEBUG"
    }, 
    "disable_existing_loggers": true, 
    "handlers": {
        "queue": {
            "queue": null, 
            "class": "myclass1.QueueHandler"
        }
    }
}, 
"number_of_archived_logs": 15, 
"log_max_size": "300M", 
"cron_job_dir": "/etc/cron.hourly/", 
"logs_dir": "/var/log/patternex/", 
"log_rotate_dir": "/etc/logrotate.d/"
}

And we want to update the queue class, the path of the key would be - log_config_worker.handlers.queue.class

We can use the following function to update the value:

def get_updated_dict(obj, path, value):
    key_list = path.split(".")

    for k in key_list[:-1]:
        obj = obj[k]

    obj[key_list[-1]] = value

get_updated_dict(data, "log_config_worker.handlers.queue.class", "myclass2.QueueHandler")

This would update the dictionary correctly.

I made a simple function, in which you give the key, the new value and the dictionary as input, and it recursively updates it with the value:

def update(key,value,dictionary):
    if key in dictionary.keys():
        dictionary[key] = value
        return
    dic_aux = []
    for val_aux in dictionary.values():
        if isinstance(val_aux,dict):
            dic_aux.append(val_aux)
    for i in dic_aux:
        update(key,value,i)
    for [key2,val_aux2] in dictionary.items():
        if isinstance(val_aux2,dict):
            dictionary[key2] = val_aux2

dictionary1={'level1':{'level2':{'levelA':0,'levelB':1}}}
update('levelB',10,dictionary1)
print(dictionary1)

#output: {'level1': {'level2': {'levelA': 0, 'levelB': 10}}}

Hope it answers.

If you want to replace a "full nested dictionary with arrays" you can use this snippet :

It will replace any "old_value" by "new_value". It's roughly doing a depth-first rebuilding of the dictionary. It can even work with List or Str/int given as input parameter of first level.

def update_values_dict(original_dict, future_dict, old_value, new_value):
    # Recursively updates values of a nested dict by performing recursive calls

    if isinstance(original_dict, Dict):
        # It's a dict
        tmp_dict = {}
        for key, value in original_dict.items():
            tmp_dict[key] = update_values_dict(value, future_dict, old_value, new_value)
        return tmp_dict
    elif isinstance(original_dict, List):
        # It's a List
        tmp_list = []
        for i in original_dict:
            tmp_list.append(update_values_dict(i, future_dict, old_value, new_value))
        return tmp_list
    else:
        # It's not a dict, maybe a int, a string, etc.
        return original_dict if original_dict != old_value else new_value

Yes! And another solution. My solution differs in the keys that are being checked. In all other solutions we only look at the keys in dict_b. But here we look in the union of both dictionaries.

Do with it as you please

def update_nested(dict_a, dict_b):
    set_keys = set(dict_a.keys()).union(set(dict_b.keys()))
    for k in set_keys:
        v = dict_a.get(k)
        if isinstance(v, dict):
            new_dict = dict_b.get(k, None)
            if new_dict:
                update_nested(v, new_dict)
        else:
            new_value = dict_b.get(k, None)
            if new_value:
                dict_a[k] = new_value

Another way of using recursion:

def updateDict(dict1,dict2):
    keys1 = list(dict1.keys())
    keys2= list(dict2.keys())
    keys2 = [x for x in keys2 if x in keys1]
    for x in keys2:
        if (x in keys1) & (type(dict1[x]) is dict) & (type(dict2[x]) is dict):
            updateDict(dict1[x],dict2[x])
        else:
            dict1.update({x:dict2[x]})
    return(dict1)

Credit to: @Gustavo Alves Casqueiro for original answer

I honestly would have preferred using a lib that could do the heavy lifting for me, but I just couldn't find something that did what I needed.

I have only added a couple of additional checks to this function.

I have included a check for lists within a dict and added a parameter for the name of a nested dict to correctly update the nested dict KEY when there may be another KEY within the OUTER dict with the same name.

Updated function:

def update(dictionary: dict[str, any], key: str, value: any, nested_dict_name: str = None) -> dict[str, any]:
    if not nested_dict_name:  # if current (outermost) dict should be updated
        if key in dictionary.keys():  # check if key exists in current dict
            dictionary[key] = value
            return dictionary
    else:  # if nested dict should be updated
        if nested_dict_name in dictionary.keys():  # check if dict is in next layer
            if isinstance(dictionary[nested_dict_name], dict):
                if key in dictionary[nested_dict_name].keys():  # check if key exists in current dict
                    dictionary[nested_dict_name][key] = value
                    return dictionary
            if isinstance(dictionary[nested_dict_name], list):
                list_index = random.choice(range(len(dictionary[nested_dict_name])))  # pick a random dict from the list

                if key in dictionary[nested_dict_name][list_index].keys():  # check if key exists in current dict
                    dictionary[nested_dict_name][list_index][key] = value
                    return dictionary

    dic_aux = []

    # this would only run IF the above if-statement was not able to identity and update a dict
    for val_aux in dictionary.values():
        if isinstance(val_aux, dict):
            dic_aux.append(val_aux)

    # call the update function again for recursion
    for i in dic_aux:
        return update(dictionary=i, key=key, value=value, nested_dict_name=nested_dict_name)

Original dict:

{
    "level1": {
        "level2": {
            "myBool": "Original",
            "myInt": "Original"
        },
        "myInt": "Original",
        "myBool": "Original"
    },
    "myStr": "Original",
    "level3": [
        {
            "myList": "Original",
            "myInt": "Original",
            "myBool": "Original"
        }
    ],
    "level4": [
        {
            "myList": "Original",
            "myInt": "UPDATED",
            "myBool": "Original"
        }
    ],
    "level5": {
        "level6": {
            "myBool": "Original",
            "myInt": "Original"
        },
        "myInt": "Original",
        "myBool": "Original"
    }
}

Data for updating (using pytest):

@pytest.fixture(params=[(None, 'myStr', 'UPDATED'),
                        ('level1', 'myInt', 'UPDATED'),
                        ('level2', 'myBool', 'UPDATED'),
                        ('level3', 'myList', 'UPDATED'),
                        ('level4', 'myInt', 'UPDATED'),
                        ('level5', 'myBool', 'UPDATED')])
def sample_data(request):
    return request.param

The 'UPDATED' parameter doesn't make sense in this smaller use case (since I could just hard-code it), but for simplicity when reading the logs, I didn't want to see multiple data-types and just made it show me an 'UPDATED' string.

Test:

@pytest.mark.usefixtures('sample_data')
def test_this(sample_data):
    nested_dict, param, update_value = sample_data

    if nested_dict is None:
        print(f'\nDict Value: Level0\nParam: {param}\nUpdate Value: {update_value}')
    else:
        print(f'\nDict Value: {nested_dict}\nParam: {param}\nUpdate Value: {update_value}')

    # initialise data dict
    data_object = # insert data here (see example dict above)

    # first print as is
    print(f'\nOriginal Dict:\n{data_object}')

    update(dictionary=data_object,
           key=param,
           value=update_value,
           nested_dict_name=nested_dict)

    # print updated
    print(f'\nUpdated Dict:\n{data_object}')

There is one caveat, when you have a dict like this:

{
    "level1": {
        "level2": {
            "myBool": "Original"
        },
        "myBool": "Original"
    },
    "level3": {
        "level2": {
            "myBool": "Original"
        },
        "myInt": "Original"
    }
}

Where level2 is under level1 AND level3. This would require making using of a list or something with the nested_dict_name and passing in the name of the outer dict AND inner dict (['level5', 'level2']) and then somehow looping through the values to find that dict.

However, since I haven't yet ran into this issue for the data objects I use, I haven't spent the time trying to solve this "issue".

Convert your dictionaries into NestedDict

from ndicts.ndicts import NestedDict

dictionary1 = {'level1': {'level2': {'levelA': 0, 'levelB': 1}}}
update = {'level1': {'level2': {'levelB': 10}}}

nd, nd_update = NestedDict(dictionary1), NestedDict(update)

Then just use update

>>> nd.update(nd_update)
>>> nd
NestedDict({'level1': {'level2': {'levelA': 0, 'levelB': 10}}})

If you need the result as a dictionary nd.to_dict()

To install ndicts pip install ndicts

a new Q how to By a keys chain

dictionary1={'level1':{'level2':{'levelA':0,'levelB':1}},'anotherLevel1':{'anotherLevel2':{'anotherLevelA':0,'anotherLevelB':1}}}
update={'anotherLevel1':{'anotherLevel2':1014}}
dictionary1.update(update)
print dictionary1
{'level1':{'level2':{'levelA':0,'levelB':1}},'anotherLevel1':{'anotherLevel2':1014}}

you could try this, it works with lists and is pure:

def update_keys(newd, dic, mapping):
  def upsingle(d,k,v):
    if k in mapping:
      d[mapping[k]] = v
    else:
      d[k] = v
  for ekey, evalue in dic.items():
    upsingle(newd, ekey, evalue)
    if type(evalue) is dict:
      update_keys(newd, evalue, mapping)
    if type(evalue) is list:
      upsingle(newd, ekey, [update_keys({}, i, mapping) for i in evalue])
  return newd