Skip Fact/Theory when precondition is not satisfied

user_1288449 0
Viewed 62

Using xUnit 2.4.1, I'm looking for a way to make it do my twisted bidding when it comes to dynamically skipping integration tests.

I'm aware xUnit is not designed for integration testing, but I have no intention to use xUnit for some tests and Not-xUnit for other tests. I think the distinction between unit and integration tests is pretty useless for a test runner.

With that out of the way, I have some tests that depend on a database. I added traits to the tests as follows.

public class Some_database_interaction
{
    [Fact]
    [Trait("Demand", "SqlServer")]
    public async Task One_thing_is_inserted()
    {
        using var connection = await Connect();
        var applicationService = new SomeService(connection);
        
        var resultCount = await applicationService.DoTheImportantThing("some args", true);

        Assert.Equal(1, resultCount);
    }
}

Currently I'm using a command to skip tests that depend on the database when I don't want them to run.

dotnet test --filter Demand!=SqlServer

This is fine for use in a CI pipeline, but I cannot expect team members to type this command every time.

Instead I want some sort of way to detect capabilities and then skip tests if a demand is not satisfied. I don't know how to do that, though.

So far my best attempt is to add noise to my tests in the form of catching the error and short-circuiting the test.

SqlConnection connection;
try
{
    connection = await Connect();
}
catch
{
    // Exit the test
    return;
}

using connection;
var applicationService = new SomeService(connection);

But this has the undesired side-effect of showing test as successful instead of skipped.

Any hints on where to go from here? I know xUnit 3 will have Assert.Skip() but for now I am stuck on xUnit 2.

xunit c#
2 Answers

Here is another try based on How do I skip specific tests in xUnit based on current platform that I think does what you want.

It involves deriving a class from FactAttribute which can set its Skip attribute based on whether a connection is available or not.

using System;
using Xunit;

namespace SO73109781_skipping_unit_tests
{
    public class ConnectionUnitTests
    {
        [FactSkippedOnNoConnection]
        public void MyUnitTest()
        {
            Assert.Equal(1, 1);
        }
    }

    public class FactSkippedOnNoConnectionAttribute : FactAttribute
    {
        public FactSkippedOnNoConnectionAttribute()
        {
            try
            {
                // Simulate no connection by throwing here
                throw new Exception();
                // connection = await Connect();
            }
            catch
            {
                Skip = "No connection";
            }
        }
    }
}

When the method lands in the catch it sets the Skip attribute and the test is skipped and shown as such in the test explorer. When the try is successful, the test is run.

user_74240380

To see what would happen, I mocked up my second suggestion as follows:

using System;
using System.Collections;
using System.Collections.Generic;
using Xunit;

namespace SO73109781_skipping_unit_tests
{
    public class ConnectionUnitTests
    {
        [Theory]
        [ClassData(typeof(ConnectionChecker))]
        public void MyUnitTest(string connected)
        {
            Assert.Equal(1, 1);
        }
    }


    public class ConnectionChecker : IEnumerable<object[]>
    {
        private static bool? connectionAvailable = null;

        public ConnectionChecker()
        {

            if (connectionAvailable == null)
            {
                /* What you would want to do here:
                SqlConnection connection;
                try
                {
                    connection = await Connect();
                    connectionAvailable = true;
                }
                catch
                {
                    connectionAvailable = false;
                }
                */ 
            }
            // For demonstration just set to specific value
            connectionAvailable = false;
        }

        public IEnumerator<object[]> GetEnumerator()
        {
            if (connectionAvailable ?? false)
            {
                yield return new string[] { "Connected" };
            }
        }

        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    }

}

When connectionAvailable = true; is used, the unit tests show up in the explorer and can be run:

Test explorer with connectionAvailable = true

When it is false the test shows up without any data:

Test explorer with connectionAvailable = false

When you run the test it fails with System.InvalidOperationException : No data found for SO73109781_skipping_unit_tests.ConnectionUnitTests.MyUnitTest, so I suppose this doesn't satisfy your requirement that the test not show up as failed.

One final suggestion: if you're willing to bring down the whole test process, you can throw an unhandled exception in the constructor of your test class when there is no connection available. None of the tests will run, they will be marked as "not run", but any tests from other projects that had not yet run will also not run. (I discovered this by accident when my application was throwing an unhandled exception.)

user_74240380
Related