python timeit: function with decorator

user_5506400 0
Viewed 53

I am trying to figure out whether to compile regex expressions upfront, or alternativelt for startup speed, only compile they once when they are first required.

I can use timeit to test:

re.compile(r'#(?:(?:[\da-f]{3}){1,2}|(?:[\da-f]{4}){1,2})|rgb\(\d{1,3},\d{1,3},\d{1,3}\)')

I don't know know how to figure out how long the parser takes (or how to use timeit to time it) to see the following code (not execute it, just "see" it).

from functools import cache

@cache
def get_color_pattern():
    return re.compile(r'#(?:(?:[\da-f]{3}){1,2}|(?:[\da-f]{4}){1,2})|rgb\(\d{1,3},\d{1,3},\d{1,3}\)')
caching python
2 Answers

Python code is interpreted "top to bottom". Therefore, if you declare variables either side of your function that are initialised with a "timestamp" you can deduce that the difference between those values will be the time taken to parse the code between them. So, for example,

from functools import cache
import time
import re

start = time.perf_counter()
@cache
def get_color_pattern():
    return re.compile(r'#(?:(?:[\da-f]{3}){1,2}|(?:[\da-f]{4}){1,2})|rgb\(\d{1,3},\d{1,3},\d{1,3}\)')
print(time.perf_counter()-start)

start = time.perf_counter()
re.compile(r'#(?:(?:[\da-f]{3}){1,2}|(?:[\da-f]{4}){1,2})|rgb\(\d{1,3},\d{1,3},\d{1,3}\)')
print(time.perf_counter()-start)

Output:

5.995000265102135e-06
0.00024428899996564724

Thus we see the difference in time taken to parse the function versus compiling the expression

user_175803810

If you want to check how long it takes with the cache generated, just run the function once before timing to generate the cache.

from functools import cache
import re
import timeit

@cache
def get_color_pattern():
    return re.compile(r'#(?:(?:[\da-f]{3}){1,2}|(?:[\da-f]{4}){1,2})|rgb\(\d{1,3},\d{1,3},\d{1,3}\)')

# Run once to generate cache
get_color_pattern()
number = 1000

t = timeit.timeit(get_color_pattern, number=number)

print(f"Total {t} seconds for {number} iterations, {t / number} at average")

Out: Total 4.5900000000001495e-05 seconds for 1000 iterations, 4.5900000000001496e-08 at average

Running with cache commented out, seems to be roughly 10x slower: Total 0.0004936999999999997 seconds for 1000 iterations, 4.936999999999997e-07 at average

As pointed out by Kelly Bundy, re.compile already includes a built in caching feature.

user_145362150
Related