How much optimization does `in` have over `__contains__`?

Viewed 114

So in the itertools recipe section, they have a snipped of code that looks like:

seen = set()
seen_add = seen.add

I was wondering whether a similar idea might bridge some of the performance gap between in and __contains__. For instance, with the following code:

seen = set()
seen_add = seen.add
in_seen = seen.__contains__
for item in iterable:
    in_seen(item)

vs

seen = set()
seen_add = seen.add
in_seen = seen.__contains__  # make identical in beginning
for item in iterable:
    item in seen

So if I'm reading the output from dis correctly, the question comes down to "is x in y faster than func(x)?"

Edit: to those saying it doesn't matter, I'm not using this as an optimization. I'm trying to understand the language better by picking this element apart.

2 Answers

We're talking a couple dozen nanoseconds at most, so usually it doesn't matter. And, even when it does, things are more complicated than they first appear.


Pre-binding seen.__contains__ as seen_contains will speed things up over calling seen.__contains__, but not as much as just using (the more obvious and idiomatic) in seen instead.

So, why is this different than seen_adds?

In the case of seen.add(), you're explicitly creating and calling a bound method, and there's no way around that. So, creating the bound method once, instead of each time… is still usually not worth it, but in those rare cases when you need to save the nanoseconds, it's a win.

In the case of in seen, you're not explicitly creating a bound method, you're just evaluating an operator. In CPython, if seen is an instance of a Python class, that will implicitly create a bound method—but if it's an instance of a builtin class, it will just directly look up the method in the C slot and call that. So, while you save time by creating the bound method once instead of over and over, it's still not as much as the time you waste calling the C function through a bound method instead of calling it directly.

Of course in a different Python implementation—or just with a different type that wasn't a builtin—things might be different.


If this actually matters (which it usually won't), you should of course test it with the platform, Python implementation, and type that you care about.

But, purely as an example, I'll test it with 64-bit python.org CPython 3.7 on my MacBook Pro with set:

In [778]: s = {1, 2, 3}
In [779]: sc = s.__contains__
In [780]: %timeit 4 in s
33.9 ns ± 0.444 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [781]: %timeit s.__contains__(4)
69.3 ns ± 0.936 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [782]: %timeit sc(4)
47.6 ns ± 0.866 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

As expected, sc gets back some of our wasted time, but not all of it.

But with a pure-Python type:

In [787]: class Set:
     ...:     def __contains__(self, n):
     ...:         return 1 <= n < 4
In [788]: s = Set()
In [789]: sc = s.__contains__
In [790]: %timeit 4 in s
129 ns ± 5.69 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [791]: %timeit s.__contains__(4)
124 ns ± 1.14 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [792]: %timeit sc(4)
108 ns ± 1.19 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

… 4 in s is slightly slower than s.__contains__(4) (because it's basically just a wrapper around calling exactly that), and creating the bound method makes it even faster.

So, we got completely opposite results with two different types that represent the same value.

And again, the biggest difference in any of these cases is still only 35ns.


As a side note, pre-binding the method helps a little more with locals than globals. (Local variable lookup is significantly faster than attribute lookup; global variable lookup is only a tiny bit faster than an attribute lookup.) That's harder to demonstrate in a one-liner, but you should test that yourself if that's your actual intended use.


And remember, all of that is just with CPython.

When I run the exact same code in PyPy 3.5.3/5.10.1, I get 6.39/6.29/6.31ns for set and 1.52/1.51/1.50ns for Set.

Notice that almost all of the details turned out exactly the other way around: __contains__ is faster than in for set, pre-binding it actually slows things down rather than speeding them up, and the non-builtin Set is 4x faster rather than 3x slower. Why? I can make some guesses, but whenever I try to dive into PyPy's JIT for reliable answers I come out three days later having learned nothing more than that Armin Rigo is an 18th-level wizard.

(Also notice that just switching Python interpreters made an order of magnitude more difference than any micro-optimization we could do within the language.)

in does seem to be faster. At a guess, COMPARE_OP is more efficent than CALL_FUNCTION because it knows how many arguments it has.

haugh@~$ python3 -m timeit -s "l = {1}" "2 in l"
10000000 loops, best of 3: 0.029 usec per loop

haugh@~$ python3 -m timeit -s "l = {1}" "l.__contains__(2)"
10000000 loops, best of 3: 0.0612 usec per loop

haugh@~$ python3 -m timeit -s "l = {1}; isin=l.__contains__" "isin(2)"
10000000 loops, best of 3: 0.0301 usec per loop
Related