How can I avoid calling same function inside an inner pythonical style loop

user_6124796 0
Viewed 54

I have a high cpu consuming function, lets say:

def high_cost_function(p):
  ... lots of operations...
  return p

which I want to use under an inner loop in python:

Example:

paths1= ['a','b','c']
paths2= ['b','c','d']
[any(x.startswith(high_cost_function(y)) for x in paths1) for y in paths2]

Which returns properly: [True, True, False]

However, as you can see, I am forced to call high_cost_function(y) for each of the x in paths1.

If it was a normal loop, I would probably simply do:

for y in paths2:
  tmp_var = high_cost_function(y)
  for x in paths1:
    ...
    use tmp_var
    ...

Is there anyway to implement this in the first "pythonical stylish way"??

python-3.x python loops performance
3 Answers

Iterate over a (generated) list of ys to which high_cost_function has already been applied:

[any(x.startswith(y) for x in paths1) for y in map(high_cost_function, paths2)]
user_4760

The answer by deceze is already perfect for the given use case. In a more general case, when you might also need y itself, you could use a nested generator expression (creating a dict here to illustrate using both):

{y: any(x.startswith(hcf_y) for x in paths1)
 for (y, hcf_y) in ((y, high_cost_function(y)) for y in paths2)}

Or use a second loop iterating over a single-element list to sort-of define a variable inside the list-comp:

{y: any(x.startswith(hcf_y) for x in paths1)
 for y in paths2 for hcf_y in [high_cost_function(y)]}
user_16396250

If results of high_cost_function are not large, you could also use caching

from functools import lru_cache
@lru_cache(maxsize=None)
def high_cost_function(p):
  ... lots of operations...
  return p
user_46018900
Related