How to output information about failure in a forward compatible way (extension hooks)?

user_1416445 0
Viewed 67

The question I have here is about how to build in a forward compatible way the problem I am facing. Technically, I know how to do it in the current version (8.5, looking to move to 9 soon), but what is officially supported, seems to not satisfy the needs I have and my fear of course, is to be blocked in the future with my current approach.

I am currently using PHPUnit extension hooks. Pretty much following the same exact example we have in the documentation: https://phpunit.readthedocs.io/en/9.5/extending-phpunit.html For the sake of simplicity, I'll paste a slightly modified version:

<?php declare(strict_types=1);
namespace Vendor;

use PHPUnit\Runner\AfterLastTestHook;
use PHPUnit\Runner\AfterTestFailureHook;
use PHPUnit\Runner\BeforeFirstTestHook;

class TestExtension implements BeforeFirstTestHook, AfterLastTestHook, AfterTestFailureHook
{

    public function executeBeforeFirstTest(): void
    {
        echo 'Testing with configuration value';
    }

    public function executeAfterLastTest(): void
    {
        echo 'Second config value is OK!';
    }

    public function executeAfterTestFailure(string $test, string $message, float $time): void
    {
        echo "Failed test took $time seconds";
    }
}

However, this triggers an awkward result:

PHPUnit 8.5.21 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.0.11 with Xdebug 3.0.4
Configuration: tests/unit.xml

Testing with configuration value....Failed test took 0.0055649280548096 seconds..                                                              6 / 6 (100%)Second config value is OK!

Time: 2.21 seconds, Memory: 38.00 MB

OK (6 tests, 9 assertions)

What I would like to do though, is much closer to what is possible to do with an instance of PHPUnit\Util\Printer which allows to append messages or information about the test and is render as part of the result. Upon failure, I would like to be able to see something along the lines of:

PHPUnit 8.5.21 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.0.11 with Xdebug 3.0.4
Configuration: /Users/sebastian.machuca/projects/cloud-dev-vm/bcappvm/codebases/bigcommerce/tests/unit.xml

Testing with configuration value......

Time: 47.63 seconds, Memory: 132.50 MB

There were 2 errors:

1) Vendor\Tests\TestMyClass::testSomeMethod
Failed test took 0.25880694389343 seconds

The problem here is that PHPUnit\Framework\TestListener is considered to be @deprecated and PHPUnit\Util\Printer is considered to be @internal with a clear "This class is not covered by the backward compatibility promise for PHPUnit" message.

So, again, the question here is about how to build in a forward compatible way. How could achieve the equivalent behaviour that today is possible with Printer Listeners, but using non internal and non deprecated functionality?

phpunit
1 Answers

I don't know if it is possible to attach additional (structured) information to an error and extend the printer to take it into consideration.

If you're concerned about the command-line runner, a globally possible option might be to add output at the very end when the test-runner exits - at least for standard streams (e.g. stdout, standard output).

When you create the timing / error information, you pass them to another object you instantiate (e.g. on a static property or a different global variable) once that collects the messages/data.

That object's destructor is being called when PHP exits. You can then output the collected data more structured and for better reading (on standard output).

This does not answer on how to do it with Phpunit specifically, just saying so this is clearly stated and you're very much concerned about that. This is standard PHP behaviour.

For your interoperability problem, I'd wrap the functionality apart from how its triggered so that its more modular for future changes.

Next to that I'd check the Phpunit version and present a warning or similar so that if the version changes you get the chance to adopt if new functionality is available. Additionally you can pin the development utilities version with the repository (e.g. commmit composer.lock). Then it works what works and perhaps in the future there will be the interface you're looking for.

Unless you have to rely on what is available even if there isn't a backwards compatible promise for it.

And sure, there is always another option: Take part in the development in the Phpunit project and bring these things forward. Make it your own.

user_3674560
Related