How to warmup Symfony environment before a functional test to check execution time?

Viewed 531

In the functional tests of a Symfony5.2 application, I added some profiling assertions to check execution time.

    private static function assertTime(?Profile $profile, int $milliseconds = self::MAX_TIMES)
    {
        if (null === $profile) {
            static::markTestSkipped('To test time of this use case, please unable profiling');
        }

        /** @var TimeDataCollector $time */
        $time = $profile->getCollector('time');
        static::assertLessThan(
            $milliseconds,
            $time->getDuration(),
            sprintf(
                'Checks that execution time (%dms) is less than %dms (token %s)',
                $time->getDuration(),
                $milliseconds,
                $profile->getToken()
            )
        );
    }

It works fine.

But each time I launch my testsuite, the first test failed because of execution time:

symfony php ./bin/phpunit
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.6 with Xdebug 2.9.5
Configuration: /home/alexandre/PhpstormProjects/bb-one/phpunit.xml.dist

Testing 
F.........................                                        26 / 26 (100%)

Time: 00:19.352, Memory: 82.50 MB

There were 1 failure:

1) App\Tests\Functional\MailTest::testShowArchived
Checks that execution time (2952ms) is less than 1000ms (token b6742a)
Failed asserting that 2952.1640625 is less than 1000.

I know this is the first functional test, because, if I change default order, I have:

symfony php ./bin/phpunit --order-by reverse

There were 1 failure:

1) App\Tests\Functional\MailTest::testShowTrash
Checks that execution time (2868ms) is less than 1000ms (token a3342a)
Failed asserting that 2868.4316425 is less than 1000.
  • I tried to purge then warmup cache, before launching testsuite but it changes nothing.
  • I read this answer, but it isn't a good answer.
  • I tried to start the server in test mode, no change.
  • I tried to create a kernel in the bootstrap to call the console and clear the test cache (it didn't help)
  • I tried to create an initial test like AAATest::testAAA(){/*..some functional test without profiling assertion ..*/}, this solution is acceptable when we launch our full tests as you can see in the below output:
Testing 
...........................                                       27 / 27 (100%)

Time: 00:20.783, Memory: 76.50 MB

OK (27 tests, 174 assertions)

But when a developer is optimizing a specific use case of the site, he only launchs the corresponding test, and the time is increased by startup and the test failed.

I find a bad solution to avoid that the first one failed during the full testsuite. Now, I'm searching a solution to avoid that my test failed when I launch only one test!

1 Answers

We probably miss more context on the aim of your tests because I don't understand how you can collect usable results with a single run of each test to get an accurate profiling.

I think what you are looking for is of course cache warm-up as already said but you might solve your problem taking advantage of PHP preloading and its integration with Symfony. By tagging your services accordingly to your tests you will probably yield the necessary performance to pass your tests.

But again, give more context about your intentions profiling your code in tests. I suggest that you take other approaches to load testing.

Related