List location of executables within the CTest test suite

Question

I have some source code that is compiled using CMake, with unit tests that are added to CTest via the CMake directive add_test(). I want the list of executables (absolute/relative path) that are used within this test suite.

Since tests are added as follows:

add_test(NAME ${A} COMMAND ${execA})
add_test(NAME ${B} COMMAND ${execB})
add_test(NAME ${C} COMMAND ${execA} ${addOptions})

there are only two distinct executables (${execA}, ${execB}) for three tests (${A}, ${B}, and ${C}).

I am totally fine having duplicates and ignoring options or having options.

So, the ideal output would be the following (but I certainly can do some parsing manually if needed):

 src/folder1/test/testThisFunction
 src/folder2/test/testThatFunction
 src/folder1/test/testThisFunction -WithThisFlag

The closest I was able to get was with:

ctest -N,--show-only

which does not run the tests, but simply shows them:

    Start 1: testA
1/3 Test #1: testA .......................   Passed    0.01 sec
    Start 2: testB
2/3 Test #2: testB .......................   Passed    0.01 sec
    Start 3: testC
3/3 Test #3: testC ........................  Passed    0.01 sec

Unfortunately, this output does not contain the information about the path to the executable.

In this example above, it is assumed that

${execA} = testThisFunction
${execB} = testThatFunction

where testThisFunction and testThatFunction are CMake targets (unit tests) and

${A} = "testA"
${B} = "testB"
${C} = "testC"
${addOptions} = "-WithThisFlag"

store names of the tests and options, respectively.

While I have the access to CMakeLists.txt, I strongly prefer to do this solely on ctest level after CMake configuring and subsequent compilation are already completed (thus, not generating list of executables using CMake commands in CMakeLists.txt).

If this is relevant, I am using CTest 3.10.2, but open to upgrade.

user_3987854 · Answer

Firstly, you only need the -N or the --show-only option in the ctest command, not both. In your case, CTest silently ignored your command line option -N,--show-only because it was not recognized. The output indicates that your tests did run. To simply list them, use:

ctest --show-only

Getting to your question: If you upgrade to CMake 3.14 or higher, you gain the JSON-formatted ctest --show-only option.

ctest --show-only=json-v1

This will print information about each of your tests, including the arguments passed to each. Your output from this may contain something like this:

  "tests" :
  [
    {
      "backtrace" : 1,
      "command" :
      [
        "src/folder1/test/testThisFunction"
      ],
      "config" : "Debug",
      "name" : "testA",
      "properties" :
      [
        {
          "name" : "WORKING_DIRECTORY",
          "value" : "src"
        }
      ]
    },
    {
      "backtrace" : 3,
      "command" :
      [
        "src/folder2/test/testThatFunction"
      ],
      "config" : "Debug",
      "name" : "testB",
      "properties" :
      [
        {
          "name" : "WORKING_DIRECTORY",
          "value" : "src"
        }
      ]
    },
    {
      "backtrace" : 5,
      "command" :
      [
        "src/folder1/test/testThisFunction",
        "-WithThisFlag"
      ],
      "config" : "Debug",
      "name" : "testC",
      "properties" :
      [
        {
          "name" : "WORKING_DIRECTORY",
          "value" : "src"
        }
      ]
    }
  ],

In the output, the full path to the executable and the test's command line arguments are listed after "command" :.

List location of executables within the CTest test suite

1 Answers