Timeout attribute ignored

user_4849753 0
Viewed 879

I use console runner (xUnit 2.4.1) to execute set of tests. Some tests should fail in case they need too long to finish, but it seems like Timeout attribute is ignored.

This is my simple test method that should fail:

[Fact(Timeout = 1000)]
public void ShouldTimeout()
{
    Thread.Sleep(15000);
}

I execute runner with the option -parallel none in order to avoid undefined behaviour:

xunit.console.exe tests.dll -parallel none -method "Tests.TessClass.ShouldTimeout"

and I also added [assembly: CollectionBehavior(DisableTestParallelization = true)] into my assembly configuration.

The test, however, still finishes successfully with the following output.

xUnit.net Console Runner v2.4.1 (64-bit Desktop .NET 4.5.2, runtime: 4.0.30319.42000)
  Discovering: tests
  Discovered:  tests
  Starting:    tests
  Finished:    tests
=== TEST EXECUTION SUMMARY ===
   tests  Total: 1, Errors: 0, Failed: 0, Skipped: 0, Time: 15,151s

Did I miss something? Is the parallelization still not disabled and therefore Timeout is ignored?

xunit.net .net c#
2 Answers

Looks like xUnit code has some internal SynchronizationContext usage, which results in all threads blocked when synchronous wait is executed inside a test - i.e. Thread.Sleep(). See source code

But, Timeout works just fine if your test code is fully async:

[Fact(Timeout = 1000)]
public async void ShouldTimeout()
{
    await Task.Delay(3000);
}

So I see 2 options here for you: either create a bug for xUnit regarding synchronous code blocking breaking Timeout behavior or rewrite your code to be async if your code base allows it.

user_21523340

In xUnit v2, the [Timeout] attribute is no longer supported.

Details are here: Migrating unit tests from v1 to v2

Excerpt:

The support for tests that automatically time out has been removed from xUnit.net v2, and there is no direct replacement for this feature. The reason it was removed was that v2 is designed from the ground up to be async and parallel, and accurately timing tests in such a design is effectively impossible.

While I'd expect a compile time error, I note that you're using a v2 runner...

user_15673520
Related