Cannot overwrite __contains__ with a lambda

user_2765164 0
Viewed 41

I would like to create a class can be used in in statements, and the condition is passed to the object in __init__. An example:

class Set:
    def __init__(self, contains):
        self.__contains__ = contains  # or setattr; doesn't matter

top = Set(lambda _: True)
bottom = Set(lambda _: False)

The problem with this is that 3 in top returns TypeError: argument of type 'Set' is not iterable, even though top.__contains__(3) returns True as expected.

What's more, if I modify the code as such:

class Set:
    def __init__(self, contains):
        self.__contains__ = contains

    def __contains__(self, x):
        return False

top = Set(lambda _: True)

, 3 in top will return False, whereas top.__contains__(3) returns True as expected, again.

What is happening here? I am on Python 3.9.2.

(Note: the same happens with other methods that are part of the data model, such as __gt__, __eq__ , etc.)

python
2 Answers

That's because magic methods are looked up on the class, not the instance. The interpreter circumvents the usual attribute-getting mechanisms when performing "overloadable" operations.

It seems to be this way because of how it was originally implemented in CPython, for example because of how type slots work (not the __slots__ slots, that's a different thing): how + or * or other operators works on a value is decided by its class, not on per instance basis.

There's a performance benefit to this: looking up a dunder method can involve a dictionary lookup, or worse, some dynamic computations with __getattr__/__getattribute__. However, I don't know if this is the main reason it is this way.

I wasn't able to find a detailed written description, but there's a talk by Armin Ronacher on YouTube going quite in depth on this.

user_102957290

__contains__ is an instance method that takes a self arg.

https://docs.python.org/3/reference/datamodel.html#object.\_\_contains\_\_

For objects that don’t define __contains__(), the membership test first tries iteration via __iter__(), then the old sequence iteration protocol via __getitem__()

So I think what is happening in each case is:

  1. The problem with this is that 3 in top returns TypeError: argument of type 'Set' is not iterable, even though top.__contains__(3) returns True as expected.

class Set:
    def __init__(self, contains):
        self.__contains__ = contains  # or setattr; doesn't matter

top = Set(lambda _: True)

You Set class doesn't have a __contains__ method, only the instance has it. So Python doesn't recognise Set objects as implementing this protocol, so it falls back to trying the search approach via __iter__... but your Set class is not iterable.

  1. 3 in top will return False, whereas top.__contains__(3) returns True as expected, again.

class Set:
    def __init__(self, contains):
        self.__contains__ = contains

    def __contains__(self, x):
        return False

top = Set(lambda _: True)

This time your Set class does have the __contains__ method, so Python will try to use it. We can see from the behaviour that 3 in top is different from top.__contains__(3). What actually happens for 3 in top is Python does something like Set.__contains__(top, 3)

So depending how you call it you get either the method on the class, or the lambda you overrode on the instance.

user_2021680
Related