phpunit --path-coverage (branch coverage) takes 100x more time

user_1592008 0
Viewed 594

I've a Laravel project with bunch of tests. I'm using pcov to calculate code coverage, it takes about 4 minutes. But pcov doesn't support branch coverage, so I decided to use xdebug for it.

Tests execution with xdebug, with code coverage but without --path-coverage (branch coverage) takes about 8 minutes.

But tests execution with xdebug, with code coverage and with --path-coverage (branch coverage) takes more that 2 hours and can't even wait until the end:

INFO[2021-09-14 21:33:24] Executing runtests with coverage xdebug
XDEBUG_MODE=coverage
php artisan test --parallel --processes=8 --verbose --passthru=--path-coverage tests/Feature --coverage-text

Warming cache for static analysis ... done [00:00.071]

............S................................................   61 / 1180 (  5%)
.............................................................  122 / 1180 ( 10%)
.............................................................  183 / 1180 ( 15%)
.............................................................  244 / 1180 ( 20%)
....... 

INFO[2021-09-15 00:00:05] finished in 2h 26m 40.458565176s

So my question is it even normal that --path-coverage takes more than 100x times for execution?

xdebug code-coverage phpunit
2 Answers

TLDR: Yes, this is expected.

Xdebug needs to do a lot of work in order to determine this information.

  1. It needs to analyse in more detail which paths and branches exist, and store this information. Even without path coverage on, Xdebug does some analyses to find dead code — this is why it is slower than pcov, which does not also do that.
  2. Xdebug needs to overload every internal PHP instruction (opcode), so that it can determine it is the start of a new branch, and if so, record that information.
  3. Xdebug needs to collect all the branches that were seen during the execution of a function, and collate that in order to find out which path was seen, and also store that information.

All these checks, and the extra storage of data, drastically reduces performance. But that's not because Xdebug is slow, but rather because it does a lot of work.

It would be nice if it was possible to only enable path/branch coverage for specific high-impact functions and methods, but I don't think that PHPUnit has a way for that (yet).

user_5080570

Yes, this is expected and normal.

user_45167790
Related