Itertools.product buiding intermediary results in memory?

user_5133524 0
Viewed 68

I have the following code

from itertools import product
from time import sleep
def slowrange(n,t):
        for i in range(n):
                sleep(t)
                yield i

for n,w in product(slowrange(5,0.2),"AB"):
        print(n,w)

since in the itertools.product doc says that:

This function is roughly equivalent to the following code, except that the actual implementation does not build up intermediate results in memory

i would expect that i would see two lines with 0 A \n 0 B then 0.2 seconds later i would see the following iteration, however, that does not occur, after 1s all the iterations are printed, if i use the following function instead of itertools.product, it does perform the desired behavior:

def product(itertor1,iterator2):
        for s in iterator1:
                for f in iterator2:
                        yield s,f

So, what's going on?, is this a bug?, it is something wrong in the docs?

itertools python
1 Answers

The intermediate results are not built up in memory, no. That's referring to this part in the Python code:

result = [[]]
for pool in pools:
    result = [x+[y] for x in result for y in pool]

What does happen is that the input iterators are converted to tuples in a pool, the first lines:

pools = [tuple(pool) for pool in args] * repeat

The pools are not intermediate results. Your slowrange() is first consumed into a tuple (and another tuple is created from the characters in the "AB" string), and from the tuples in the pool the results are produced (using an array of indices into those tuples).

user_1002970
Related