Why are references to python values, that are function parameters, stored on the stack(frame) in CPython?

user_7483073 0
Viewed 38

Python works with reference counting. That means, if there is no more reference to a value, then the memory of that value is recycled. Or in other words. As long as there is at least one remaining reference, the obj is not deleted and the memory is not released.

Lets consider the following example:

def myfn():                    
    result = work_with(BigObj()) # reference 1 to BigObj is on the stack frame.
                                 # Not yet counting any 
                                 # reference inside of work_with function
                                 # after work_with returns: The stack frame 
                                 # and reference 1 are deleted. memory of BigObj  
                                 # is released
    return result             

def work_with(big_obj):       # here we have another reference to BigObj
    big_obj = None            # let's assume, that need more memory and we don't  
                              # need big_obj any_more
                              # the reference inside work_with is deleted. However,
                              # there is still the reference on the stack. So the                                                                           
                              # memory is not released until work_with returns
    other_big_obj = BigObj()  # we need the memory for another BigObj -> we may run  
                              # out of memory here

So my question is:

Why does CPython hold an additional reference to values which are passed to functions on the stack? Is there any special purpose behind this or is it just an "unlucky" implementation detail?

My first thought on this is: To prevent the reference count from dropping to zero. However, we have still an alive reference inside the called function. So this does not make any sense to me.

cpython reference-counting function python
1 Answers

It is the way CPython passes parameters to a function. The frame holds a reference to its argument to allow passing temporary objects. And the frame is destroyed only when the function returns, so all parameters get an additional reference during the function call.

This is the reason why the doc for sys.getrefcount says:

The count returned is generally one higher than you might expect, because it includes the (temporary) reference as an argument to getrefcount().

In fact, in the callee, the reference to the arguments is known to be a borrowed reference, meaning that the callee never has to decrement it. So when you set it to None it will not destroy the object.

A different implementation would be possible, where the callee should decrement the reference to its arguments. The benefit would be that it would allow immediate destruction of temporaries. But the drawback would be that the callee should explicitely decrement the reference count of all its parameters. At C level, ref counting is already tedious, and I assume that Python implementers made that choice for simplicity.

By the way, it only matters when you pass a large temporary object to a function which is not the most common use case.

TL/DR: IMHO there is no real rationale for preventing a function to immediately destroy a temporary, it is just a consequence of the general implementation of functions in CPython.

user_35452730
Related