How can I check how many `EXPECT_*` calls failed in one test

Viewed 689

I have an integration test that is something like:

TEST(foo, test_many_foos) {
   foo_builder sut;
   sut.set_some_params();

   sut.run();

   for (const auto foo : sut) {
      EXPECT_TRUE(some_check(foo));
   }


   // TODO: Print a summary of how many EXPECT_TRUEs failed

}

Is there a way that I can print out a summary of the results of all of the EXPECT calls at the end of the test?

3 Answers

You could enhance Google Test using a custom event listener. You can define your own listener class and have it track the number of times an EXPECT_*() call fails per test:

class ExpectListener : public ::testing::EmptyTestEventListener {
    int nFailures;

    // Called before a test starts.
    virtual void OnTestStart(const ::testing::TestInfo& test_info) {
        nFailures = 0;

        printf("*** Test %s.%s starting.\n",
               test_info.test_case_name(),
               test_info.name());
    }

    // Called after a failed assertion or a SUCCEED() invocation.
    virtual void OnTestPartResult(const ::testing::TestPartResult& test_part_result) {
        if (test_part_result.nonfatally_failed()) {nFailures++;}

        printf("%s in %s:%d\n%s\n",
               test_part_result.failed() ? "*** Failure" : "Success",
               test_part_result.file_name(),
               test_part_result.line_number(),
               test_part_result.summary());
    }

    // Called after a test ends.
    virtual void OnTestEnd(const ::testing::TestInfo& test_info) {
        printf("*** Test %s.%s ending with %d failures.\n",
               test_info.test_case_name(),
               test_info.name(),
               nFailures);
    }
};

Now just replace Google Test's default listener with this custom listener:

int main(int argc, char** argv) {
    ::testing::InitGoogleTest(&argc, argv);

    ::testing::TestEventListeners& listeners = ::testing::UnitTest::GetInstance()->listeners();
    delete listeners.Release(listeners.default_result_printer());
    listeners.Append(new ExpectListener);

    return RUN_ALL_TESTS();
}

You only need to set this up once (in main()); all tests will then track the number of non-fatal failures they've experienced. Of course, you can further customize this listener if you want to tweak the test messages or track more information.

A solution that I came up with (that I'm not entirely happy with) is to do the following:

class stream_counter
{
public:
    explicit stream_counter(int *val) : val_(val) {}
    void                         increment_count() const { ++*val_; }

private:
    int *val_;
};

std::ostream &operator<<(std::ostream &os, const stream_counter &sc)
{
    sc.increment_count();
    return os;
}

TEST(foo, test_many_foos) {
   foo_builder sut;
   sut.set_some_params();

   sut.run();

   int n_failures = 0;

   for (const auto foo : sut) {
      EXPECT_TRUE(some_check(foo)) << stream_counter{&n_failures};
   }

   if(n_failures > 0) {
      std::cout << "There were " << n_failures << " failures" << std::endl;
   }

}

This abuses the operator << behaviour iof EXPECT_* which allow printing a message to the error stream if the check fails. stream_counter just wraps an integer and increments it if the object is streamed to a std::ostream

Not the cleanest solution, but it does what I want.

TEST(foo, test_many_foos) {
   foo_builder sut;
   sut.set_some_params();

   sut.run();

   auto n_failures = 0;
   for (const auto foo : sut) {
      auto const result = some_check(foo);
      EXPECT_TRUE(result);
      if (!result) ++n_failures;
   }

   if(n_failures > 0) {
      std::cout << "There were " << n_failures << " failures" << std::endl;
   }
}

This, however, means that EXPECT_TRUE() will print something about "result" rather than "some_check(foo)".

Related