What's the point of using [object instance].__self__?

user_1767047 0
Viewed 2066

I was checking the code of the toolz library's groupby function in Python and I found this:

def groupby(key, seq):
    """ Group a collection by a key function
    """
    if not callable(key):
        key = getter(key)
    d = collections.defaultdict(lambda: [].append)
    for item in seq:
        d[key(item)](item)
    rv = {}
    for k, v in d.items():
        rv[k] = v.__self__
    return rv

Is there any reason to use rv[k] = v.__self__ instead of rv[k] = v?

python
2 Answers

This is a somewhat confusing trick to save a small amount of time:

We are creating a defaultdict with a factory function that returns a bound append method of a new list instance with [].append. Then we can just do d[key(item)](item) instead of d[key(item)].append(item) like we would have if we create a defaultdict that contains lists. If we don't lookup append everytime, we gain a small amount of time.

But now the dict contains bound methods instead of the lists, so we have to get the original list instance back via __self__.

__self__ is an attribute described for instance methods that returns the original instance. You can verify that with this for example:

>>> a = []
>>> a.append.__self__ is a
True
user_84729760

This is a somewhat convoluted, but possibly more efficient approach to creating and using a defaultdict of lists.

First, remember that the default item is lambda: [].append. This means create a new list, and store a bound append method in the dictionary. This saves you a method bind on every further append to the same key, and the garbage collect that follows. For example, the following more standard approach is less efficient:

d = collections.defaultdict(list)
for item in seq:
    d[key(item)].append(item)

The problem then becomes how to get the original lists back out of the dictionary, since the reference is not stored explicitly. Luckily, bound methods have a __self__ attribute which does just that. Here, [].append.__self__ is a reference to the original [].

As a side note, the last loop could be a comprehension:

return {k: v.__self__ for k, v in d.items()}
user_29887300
Related