Pass a test based on its execution time

user_3512538 0
Viewed 372

Does pytest have something like a timer test?

i.e.

@pytest.mark.timer(60)
def test_this_code_runs_in_less_than_x():
    bla()

which will fail if bla executes for more than 60 seconds?

I can implement this easily with my own decorator, e.g.:

import time, threading                                                                                                                                                                                                     
def timeout(func):                                                                                                                                                                                              
    def wrapper():                                                                                                                                                                                              
        start_time = time.time()                                                                                                                                                                                
        timeout = start_time + 1                                                                                                                                                                                
                                                                                                                                                                                                                
        th = threading.Thread(target=func)                                                                                                                                                                      
        th.start()                                                                                                                                                                                              
        while time.time() < timeout and th.isAlive():                                                                                                                                                           
            pass                                                                                                                                                                                                
        if th.isAlive():                                                                                                                                                                                        
            raise TimeoutError()                                                                                                                                                                                
        th.join()                                                                                                                                                                                               
                                                                                                                                                                                                                
    return wrapper                                                                                                                                                                                              
                                                                                                                                                                                                                
                                                                                                                                                                                                                
@timeout                                                                                                                                                                                                        
def _test():                                                                                                                                                                                                    
    time.sleep(2)                                                                                                                                                                                               
                                                                                                                                                                                                                
                                                                                                                                                                                                                
def test_should_fail_after_one_second():                                                                                                                                                                        
    _test()

but I'm not looking to invent the wheel...

I know pytest can have a --timeout param, but I'm looking for something to be a part of the test definition, not a configurable parameter that affects ALL tests

pytest python
1 Answers

With the pytest-timeout plugin, you can mark individual tests with a timeout period, measured in seconds:

@pytest.mark.timeout(60)
def test_foo():
    pass

You can explicitly choose between the following two "thread methods" by adding a method argument to the decorator:

  • thread: runs a sleep in a separate thread and terminates the test process when the timeout is reached. Beware that this may cause issues with normal JUnit XML output or fixture teardown.
  • signal: schedules a SIGALRM alarm that cancels the test when the timeout is reached. This allows your test run to complete normally. However, if your code already uses SIGALRM, this may interfere with its normal functionality.

If your system supports signal.SIGALRM (typically Unix), the signal thread method will be used by default. Otherwise, the thread method will be used instead.

user_57527300
Related