Python: Why does copying the function name to a local namespace result in a faster access

Viewed 172

From the code here: https://www.learnsteps.com/increasing-performance-python-code/

import datetime 
alist = [str(x) for x in range(100000000)]

print("\nStandard loop.") 
a = datetime.datetime.now() 
result = [] 
for item in alist: 
    result.append(len(item)) 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 

print("\nStandard loop with function name in local namespace.") 
a = datetime.datetime.now() 
result = [] 
fn = len 
for item in alist:
    result.append(fn(item))
b = datetime.datetime.now()
print((b-a).total_seconds())

print("\nUsing map.")
a = datetime.datetime.now()
result = list(map(len, alist))
b = datetime.datetime.now()
print((b-a).total_seconds())

print("\nUsing map with function name in local namespace.")
a = datetime.datetime.now() 
fn = len 
result = list(map(fn, alist)) 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 

print("\nList comprehension.") 
a = datetime.datetime.now() 
result = [len(i) for i in alist] 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 
print("\nList comprehension with name in local namespace.") 

a = datetime.datetime.now() 
fn = len 
result = [fn(i) for i in alist] 
b = datetime.datetime.now() 
print((b-a).total_seconds())

which produces this output:

Standard loop.
20.862797

Standard loop with function name in local namespace.
16.34087

Using map.
6.893764

Using map with function name in local namespace.
6.774654

List comprehension.
9.362831

List comprehension with name in local namespace.
10.007393

Can someone provide a better explanation than 'function lookups are costly' as to why creating a function prototype close to the use of the function is somehow faster? (This doesn't work for most functions, and usually only in tight loops, but why does this happen at all?)

1 Answers

This is because name resolution starts from the local namespace first, and if it is not found locally, it would then be looked up in the next nearest enclosing code block, and then its next nearest enclosing code block, until the module code block, which is the global namespace, and if the name isn't found in the global namespace, then and only then will the interpreter look up the built-in names. This is why assigning the reference to the built-in name len to a global name fn would speed up the name resolution in your code example.

Related