Is it possible to find the seed for GoogleTest to run a specific order if the order is known?

user_3149816 0
Viewed 276

I have been given a list of 25 tests and a order to run them in that is supposedly causing a failure in the last test. Unfortunately the tool that was used to find this failure does not save the seed that was used by GoogleTests shuffle to trigger this order.

So I can of course just run the tests with shuffle and repeat a huge number of times until this order is triggered, but there is quite a lot of permutations to work through. As far as I can tell there isn't a way to just say "Hey run these tests in this order" with GoogleTest aside from knowing the seed to trigger the order.

I'm hoping there might be a way to find that seed without actually running the tests - ideally something that could be given the desired order and output the seed, but really just something that given the gtest_filter value just dumps out massive amounts of potential test orders and the seeds that trigger them would be fine.

Is there any options out there, or I'm I just running with shuffle until I get it?

googletest c++
1 Answers

There is no such option to specify the order of the tests.

However, By default (i.e. if --gtest_shuffle is not enabled), they run in the declaration order.

I can think of three ways of achieving what you want:

  1. Use your favorite scripting language (e.g. bash, python, nodejs, etc) and regular expressions to modify the original order of the tests to your desired order and create a new test file which then you run without --gtest_shuffle.

  2. Hack into Google test's source code. You can modify the shuffle function to shuffle the code in the order that you want.

  3. Look into Registering tests programmatically. I think you might be able to register your tests one by one in the order that you want, and I think Googletest will observe that order. But this also requires processing your original test file.

Obviously, none of these methods are ideal, but they might work for a one-off experiment to find the order that you want.

user_13833560
Related