What is the Python rule about "unpacking" a generator

user_1112406 0
Viewed 147

Here's a simple bounded generator.

def bounded_naturals(limit):
    num = 1
    while num <= limit:
        yield num
        num += 1

If I write

bn_gen = bounded_naturals(3)

bn_gen will be a generator object as expected.

But if I write

(a, b, c) = bounded_naturals(3)

a, b, and c will be 1, 2, and 3 respectively. That strikes me as strange since there seems to be nothing in the code that asks the generator to produce values. Is there a place in the Python specification that requires this interpretation?

Even more striking, if I write

bn_gen = (a, b, c) = bounded_naturals(3)

I get both results! bn_gen will be a generator object, and a, b, and c will be 1, 2, and 3. How should I understand what's going on?

Finally, if I write

(a, b) = bounded_naturals(3)

I get: ValueError: too many values to unpack (expected 2).

If the compiler is clever enough to do these other tricks, why isn't it clever enough to ask the generator for only as many elements as needed in this case?

Is there a section in the Python documentation that explains all this?

Thanks.

python
2 Answers

Unpacking operates on arbitrary iterables, not sequences, and it does so by iteration. When you do

(a, b, c) = bounded_naturals(3)

you are asking Python to iterate over bounded_naturals(3) and assign the results to a, b, and c.

A multiple assignment like

bn_gen = (a, b, c) = bounded_naturals(3)

works by assigning the RHS to each assignment target from left to right (not right to left like some other languages). The generator first gets assigned to bn_gen, then to (a, b, c). Note that the unpacking exhausts the generator, so iterating over bn_gen will give you nothing.

When you do

(a, b) = bounded_naturals(3)

the failure has nothing to do with Python not being clever. Python will not silently discard extra values, as that would only hide bugs. The iterable must give exactly as many elements as the unpacking requests.

Remember, just because some code could be given a non-error meaning, does not mean it should.

All of this is documented in the assignment statement docs.

user_23571120

The documentation you're looking is here:

assignment_stmt ::=  (target_list "=")+ (starred_expression | yield_expression)
...

Assignment of an object to a target list, optionally enclosed in parentheses or square brackets, is recursively defined as follows.

  • If the target list is a single target with no trailing comma, optionally in parentheses, the object is assigned to that target.
  • Else: The object must be an iterable with the same number of items as there are targets in the target list, and the items are assigned, from left to right, to the corresponding targets.

I assume you're aware that a generator is an iterable. Let's see the code

bn_gen = (a, b, c) = bounded_naturals(3)

There are two target_lists here: One single target bn_gen and three targets (a, b, c). Since bn_gen is a single target, whatever bounded_naturals(3) returns is assigned to it (in this case the generator itself). Then (a, b, c) is seen as a target list with multiple targets, the RHS (right-hand side) is "unpacked" and assigned one-by-one.

So yeah, as @Pranav Hosangadi noted in comments, you can only unpack nothing or everything.

user_59584550
Related