By default nunit tests run alphabetically. Does anyone know of any way to set the execution order? Does an attribute exist for this?
By default nunit tests run alphabetically. Does anyone know of any way to set the execution order? Does an attribute exist for this?
Your unit tests should each be able to run independently and stand alone. If they satisfy this criterion then the order does not matter.
There are occasions however where you will want to run certain tests first. A typical example is in a Continuous Integration situation where some tests are longer running than others. We use the category attribute so that we can run the tests which use mocking ahead of the tests which use the database.
i.e. put this at the start of your quick tests
[Category("QuickTests")]
Where you have tests which are dependant on certain environmental conditions, consider the TestFixtureSetUp and TestFixtureTearDown attributes, which allow you to mark methods to be executed before and after your tests.
Wanting the tests to run in a specific order does not mean that the tests are dependent on each other - I'm working on a TDD project at the moment, and being a good TDDer I've mocked/stubbed everything, but it would make it more readable if I could specify the order which the tests results are displayed - thematically instead of alphabetically. So far the only thing I can think of is to prepend a_ b_ c_ to classes to classes, namespaces and methods. (Not nice) I think a [TestOrderAttribute] attribute would be nice - not stricly followed by the framework, but a hint so we can achieve this
Usually Unit Test should be independent, but if you must, then you can name your methods in alphabetical order ex:
[Test]
public void Add_Users(){}
[Test]
public void Add_UsersB(){}
[Test]
public void Process_Users(){}
or you can do..
private void Add_Users(){}
private void Add_UsersB(){}
[Test]
public void Process_Users()
{
Add_Users();
Add_UsersB();
// more code
}
You should not depend on the order in which the test framework picks tests for execution. Tests should be isolated and independent. In that they should not depend on some other test setting the stage for them or cleaning up after them. They should also produce the same result irrespective of the order of the execution of tests (for a given snapshot of the SUT)
I did a bit of googling. As usual, some people have resorted to sneaky tricks (instead of solving the underlying testability/design issue
See Also: characteristics of a good test
In case of using TestCaseSource the key is to override string ToString method, How that works:
Assume you have TestCase class
public class TestCase
{
public string Name { get; set; }
public int Input { get; set; }
public int Expected { get; set; }
}
And a list of TestCases:
private static IEnumerable<TestCase> TestSource()
{
return new List<TestCase>
{
new TestCase()
{
Name = "Test 1",
Input = 2,
Expected = 4
},
new TestCase()
{
Name = "Test 2",
Input = 4,
Expected = 16
},
new TestCase()
{
Name = "Test 3",
Input = 10,
Expected = 100
}
};
}
Now lets use it with a Test method and see what happen:
[TestCaseSource(nameof(TestSource))]
public void MethodXTest(TestCase testCase)
{
var x = Power(testCase.Input);
x.ShouldBe(testCase.Expected);
}
This will not test in order and the output will be like this:
So if we added override string ToString to our class like:
public class TestCase
{
public string Name { get; set; }
public int Input { get; set; }
public int Expected { get; set; }
public override string ToString()
{
return Name;
}
}
The result will change and we get the order and name of test like:
Note: