Memory management for functions and decorators

user_9415264 0
Viewed 73

My question starts with how decorators work in python. Let's look at the following code:

def decorator(F):
    def wrapper():
        print("start")
        F()
        print("end")
    return wrapper

def f1():
    print("f1")

decorated_f1 = decorator(f1)
decorated_f1()

It prints,

start
f1
end

First of all, as much as I know, python uses lazy evaluation. Therefore, it does not evaluate F(), until it is required (when actually decorated_f1() is called). By then, the scope of argument is F is over (end of the decorator function). I would like to know, what python stores in memory when a function is created and overall the memory management that happens for decorators.

The second part of my question is about the results that I get after running the following codes,

def decorator(F):
    def wrapper():
        print("start")
        F()
        print("end")
    return wrapper

def f1():
    print("f1")

decorated_f1 = decorator(f1)


def f1():
    print("new f1")

decorated_new_f1 = decorator(f1)

decorated_f1()
decorated_new_f1()

it results in,

start
f1
end
start
new f1
end

However, the following code

def f1():
    print("f1")

F = f1
def wrapper():
     print("start")
     F()
     print("end")
wrapped_f1 = wrapper



def f1():
    print("new f1")

F = f1
def wrapper():
     print("start")
     F()
     print("end")

wrapped_new_f1 = wrapper

wrapped_f1()
wrapped_new_f1()

produces,

start
new f1
end
start
new f1
end

This makes me confused, because I thought these two codes should be very similar in output. That is why I need help for clarifying what and how things are stored in memory when functions or decorators are declared in python.

memory-management python-3.x python
1 Answers

You are correct that the second and third example are quite close in how they work. However there is one key difference. The scope the functions are evaluated in. The scope in this case can be seen as all the variables some place in the code has access too, and where those come from. You might have heard of the global scope, this is the scope all code is evaluated in and all variables that are global are accessible by any part of the code. There are also function scopes. These are the variables defined inside of the function, and other functions won't have access to these variables.

To further understand the difference there is one more think you should know. Functions are a reference type in Python, this means that when you declare one all later uses reference the value rather then directly accessing the value. This is the same as for lists and dictionaries.

Why does this all matter? Well because a decorator stores the function scope to correctly evaluate later. This includes the current value of F in this function

def decorator(F):
    def wrapper():
        print("start")
        F()
        print("end")
    return wrapper

However, if you don't pass F as an argument then the decorator will store the function as a reference to the global function. Thus when you chance F in the third example the F() in wrapped_f1 also changes. This explains the output you see.

Lastly, as a tip. There is a specific syntax for using decorators on functions, and it might make using them easier. Usually a decorator would look like this.

def decorator(F):
    def wrapper():
        print("start decorating")
        F()
        print("End decorating")
    return wrapper

def undecorated_func():
    print("Hellor world!")


@decorator
def decorated_func():
    print("Hello World!")


undecorated_func()
decorated_func()

Then the output is

Hello world!
Start decorating
Hello World!
End decorating
user_167007540
Related