How can I make my JUnit tests run in random order?

user_87987 0
Viewed 11213

I have the classical structure for tests, I have a test suite of different suites like DatabaseTests, UnitTests etc. Sometimes those suites contains other suites like SlowDatabaseTests, FastDatabaseTests etc.

What I want is to randomize the running order of tests so I will make sure they are not dependent to each other. Randomization should be at every level, like suite should shuffle test class order, and test class should shuffle test method order.

If it is possible to do this in Eclipse that will be the best.

junit
6 Answers

This issue is open on JUnit GitHub since 2 years, and point out 2 independent issues:
- Tests depending on the execution order;
- Non repeatable tests.

Consider adressing the issue at the root, rather than trying to use the framework to do the job afterwards. Use setUp and tearDown method to guarantee isolation, and test at the smallest level.

user_97175410

In JUnit 4.13, to run the tests within a test class in random order, write a small helper class:

import org.junit.runner.manipulation.Ordering;

import java.util.Random;

public class RandomOrder implements Ordering.Factory {
    @Override
    public Ordering create(Ordering.Context context) {
        long seed = new Random().nextLong();
        System.out.println("RandomOrder: seed = " + seed);
        return Ordering.shuffledBy(new Random(seed));
    }
}

Then, annotate your test class with:

@OrderWith(RandomOrder.class)

This way, the test methods of this one class are run in random order. Plus, if they unexpectedly fail, you know the random seed to repeat exactly this order.

I don't know though how to configure this for a whole project or a test suite.

user_164533920
Related