Calculate difference in keys contained in two Python dictionaries

Viewed 161768

Suppose I have two Python dictionaries - dictA and dictB. I need to find out if there are any keys which are present in dictB but not in dictA. What is the fastest way to go about it?

Should I convert the dictionary keys into a set and then go about?

Interested in knowing your thoughts...


Thanks for your responses.

Apologies for not stating my question properly. My scenario is like this - I have a dictA which can be the same as dictB or may have some keys missing as compared to dictB or else the value of some keys might be different which has to be set to that of dictA key's value.

Problem is the dictionary has no standard and can have values which can be dict of dict.

Say

dictA={'key1':a, 'key2':b, 'key3':{'key11':cc, 'key12':dd}, 'key4':{'key111':{....}}}
dictB={'key1':a, 'key2:':newb, 'key3':{'key11':cc, 'key12':newdd, 'key13':ee}.......

So 'key2' value has to be reset to the new value and 'key13' has to be added inside the dict. The key value does not have a fixed format. It can be a simple value or a dict or a dict of dict.

21 Answers

You can use set operations on the keys:

diff = set(dictb.keys()) - set(dicta.keys())

Here is a class to find all the possibilities: what was added, what was removed, which key-value pairs are the same, and which key-value pairs are changed.

class DictDiffer(object):
    """
    Calculate the difference between two dictionaries as:
    (1) items added
    (2) items removed
    (3) keys same in both but changed values
    (4) keys same in both and unchanged values
    """
    def __init__(self, current_dict, past_dict):
        self.current_dict, self.past_dict = current_dict, past_dict
        self.set_current, self.set_past = set(current_dict.keys()), set(past_dict.keys())
        self.intersect = self.set_current.intersection(self.set_past)
    def added(self):
        return self.set_current - self.intersect 
    def removed(self):
        return self.set_past - self.intersect 
    def changed(self):
        return set(o for o in self.intersect if self.past_dict[o] != self.current_dict[o])
    def unchanged(self):
        return set(o for o in self.intersect if self.past_dict[o] == self.current_dict[o])

Here is some sample output:

>>> a = {'a': 1, 'b': 1, 'c': 0}
>>> b = {'a': 1, 'b': 2, 'd': 0}
>>> d = DictDiffer(b, a)
>>> print "Added:", d.added()
Added: set(['d'])
>>> print "Removed:", d.removed()
Removed: set(['c'])
>>> print "Changed:", d.changed()
Changed: set(['b'])
>>> print "Unchanged:", d.unchanged()
Unchanged: set(['a'])

Available as a github repo: https://github.com/hughdbrown/dictdiffer

not sure whether its "fast" or not, but normally, one can do this

dicta = {"a":1,"b":2,"c":3,"d":4}
dictb = {"a":1,"d":2}
for key in dicta.keys():
    if not key in dictb:
        print key

As Alex Martelli wrote, if you simply want to check if any key in B is not in A, any(True for k in dictB if k not in dictA) would be the way to go.

To find the keys that are missing:

diff = set(dictB)-set(dictA) #sets

C:\Dokumente und Einstellungen\thc>python -m timeit -s "dictA =    
dict(zip(range(1000),range
(1000))); dictB = dict(zip(range(0,2000,2),range(1000)))" "diff=set(dictB)-set(dictA)"
10000 loops, best of 3: 107 usec per loop

diff = [ k for k in dictB if k not in dictA ] #lc

C:\Dokumente und Einstellungen\thc>python -m timeit -s "dictA = 
dict(zip(range(1000),range
(1000))); dictB = dict(zip(range(0,2000,2),range(1000)))" "diff=[ k for k in dictB if
k not in dictA ]"
10000 loops, best of 3: 95.9 usec per loop

So those two solutions are pretty much the same speed.

If you really mean exactly what you say (that you only need to find out IF "there are any keys" in B and not in A, not WHICH ONES might those be if any), the fastest way should be:

if any(True for k in dictB if k not in dictA): ...

If you actually need to find out WHICH KEYS, if any, are in B and not in A, and not just "IF" there are such keys, then existing answers are quite appropriate (but I do suggest more precision in future questions if that's indeed what you mean;-).

The top answer by hughdbrown suggests using set difference, which is definitely the best approach:

diff = set(dictb.keys()) - set(dicta.keys())

The problem with this code is that it builds two lists just to create two sets, so it's wasting 4N time and 2N space. It's also a bit more complicated than it needs to be.

Usually, this is not a big deal, but if it is:

diff = dictb.keys() - dicta

Python 2

In Python 2, keys() returns a list of the keys, not a KeysView. So you have to ask for viewkeys() directly.

diff = dictb.viewkeys() - dicta

For dual-version 2.7/3.x code, you're hopefully using six or something similar, so you can use six.viewkeys(dictb):

diff = six.viewkeys(dictb) - dicta

In 2.4-2.6, there is no KeysView. But you can at least cut the cost from 4N to N by building your left set directly out of an iterator, instead of building a list first:

diff = set(dictb) - dicta

Items

I have a dictA which can be the same as dictB or may have some keys missing as compared to dictB or else the value of some keys might be different

So you really don't need to compare the keys, but the items. An ItemsView is only a Set if the values are hashable, like strings. If they are, it's easy:

diff = dictb.items() - dicta.items()

Recursive diff

Although the question isn't directly asking for a recursive diff, some of the example values are dicts, and it appears the expected output does recursively diff them. There are already multiple answers here showing how to do that.

Here's a way that will work, allows for keys that evaluate to False, and still uses a generator expression to fall out early if possible. It's not exceptionally pretty though.

any(map(lambda x: True, (k for k in b if k not in a)))

EDIT:

THC4k posted a reply to my comment on another answer. Here's a better, prettier way to do the above:

any(True for k in b if k not in a)

Not sure how that never crossed my mind...

Related