GoogleTest: Increase precision of EXPECT_EQ float failures

user_1245420 0
Viewed 189

I'm seeing failures of an EXPECT_EQ for 32-bit float comparisons that look very odd:

RawDataConverter_Test.cpp:325: Failure
Expected equality of these values:
  expectedResult
Which is: 0.0726339
  actualResult
Which is: 0.0726339

I know that floating point computations performed two different ways that "mathematically ought to produce the same results" often don't produce exactly the same results due to rounding of intermediate terms (which is why GoogleTest recommends using EXPECT_FLOAT_EQ, EXPECT_DOUBLE_EQ, or EXPECT_NEAR for floating point tests), but that's not my issue here. Both of my "ways of computing" the result are deterministically identical, so they should always produce the same result; if they don't, then someone may have bungled a code change in the production version of the code.

The numbers above look the same to me, so it seemed odd that the EXPECT_EQ would fail. Then I added my own printf of the values before the EXPECT_EQ:

        float expectedResult = ...
        float actualResult = ...

        printf("\nexp=%0.20f (0x%08x)\nact=%0.20f (0x%08x) (%s)\n",
               expectedResult,
               expectedResultU32,
               actualResult,
               actualResultU32,
               (expectedResult == actualResult) ? "same" : "diff");
        EXPECT_EQ(expectedResult, actualResult);

...and now I see this:

exp=0.07263389974832534800 (0x3d94c115)
act=0.07263390719890594500 (0x3d94c116) (diff)
RawDataConverter_Test.cpp:325: Failure
Expected equality of these values:
  expectedResult
Which is: 0.0726339
  actualResult
Which is: 0.0726339

Now it's clear why the EXPECT_EQ failed, but it would be nice if GoogleTest's EXPECT_EQ failure output for float comparisons showed sufficient precision in the first place.

I know you can customize GoogleTest failure output to print custom strings for your own classes by providing a PrintTo() function or an overload of std::ostream& operator<<(std::ostream&, const T&), but is there a way to provide a custom PrintTo() or operator<<() for builtin types like float? Or is there any other way to make GoogleTest print float with greater precision than normal?

I tried doing that...

namespace testing
{
   void PrintTo(const float value, std::ostream* os)
   {
      std::streamsize oldPrecision = os->precision();
      os->precision(12);
      *os << value;
      os->precision(oldPrecision);
   }
}

...(with and without the namespace testing {}), but it was never called. I also tried...

// (same error if 2nd argument is ------>  const float &value)
std::ostream& operator<<(std::ostream& os, const float value)
{
   std::streamsize oldPrecision = os.precision();
   os.precision(12);
   os << value;   // <------- error: ambiguous overload for 'operator<<'
   os.precision(oldPrecision);
   return os;
}

...but that doesn't compile for the error reason shown in the comment.

BTW I am using GoogleTest release 1.10.0, released 03 October 2019.

googletest c++
2 Answers

I can think of several ways to get to what you want or to something close to what you want:

  1. Use EXPECT_TRUE with predicate.
  2. Use EXPECT_PRED_FORMAT with predicate (this one can produce your exact output)
  3. Print extra info with EXPECT_EQ, not exact, but close.
  4. Create a custom matcher, again, not exact, but close.

Here is the code:

#include <sstream>
#include <string>

#include "gmock/gmock.h"
#include "gtest/gtest.h"

//-----------------------------------------------------------------------------
// Use EXPECT_TRUE With Predicate
//-----------------------------------------------------------------------------
testing::AssertionResult MyFloatEq(float m, float n) {
  if (m == n) return testing::AssertionSuccess();

  return testing::AssertionFailure() << "because " << m << " != " << n;
}
TEST(FloatComparison, UsePred) {
  float expectedResult = 0.07263389974832534800;
  float actualResult = 0.07263390719890594500;

  EXPECT_TRUE(MyFloatEq(expectedResult, actualResult));
  std::cout << "----------------------------------------------------"
            << std::endl;
}
//-----------------------------------------------------------------------------
// Use EXPECT_PRED_FORMAT*
//-----------------------------------------------------------------------------
testing::AssertionResult MyFloatEq2(const char* m_expr, const char* n_expr,
                                    float m, float n) {
  if (m == n) return testing::AssertionSuccess();

  return testing::AssertionFailure()
         << "Expected equality of these values:\n"
         << "  " << m_expr << "\n    Which is: " << m << "\n"
         << "  " << n_expr << "\n    Which is: " << n << "\n";
}
TEST(FloatComparison, UsePredFormat) {
  float expectedResult = 0.07263389974832534800;
  float actualResult = 0.07263390719890594500;

  EXPECT_PRED_FORMAT2(MyFloatEq2, expectedResult, actualResult);
  std::cout << "----------------------------------------------------"
            << std::endl;
}
//-----------------------------------------------------------------------------
// Print extra info
//-----------------------------------------------------------------------------
std::string DisplayFloatEq(const float expectedResult,
                           const float actualResult) {
  std::stringstream ss;
  ss << "( displayed with high precision: " << std::setprecision(12)
     << expectedResult << ", " << actualResult << " )";

  return ss.str();
}

TEST(FloatComparison, PrintExtraInfo) {
  float expectedResult = 0.07263389974832534800;
  float actualResult = 0.07263390719890594500;

  EXPECT_EQ(expectedResult, actualResult)
      << DisplayFloatEq(expectedResult, actualResult);

  std::cout << "----------------------------------------------------"
            << std::endl;
}
//-----------------------------------------------------------------------------
// Use a Custom Matcher
//-----------------------------------------------------------------------------
MATCHER_P(IsFloatEq, a,
          (std::string(negation ? "should not be equal to "
                                : "should be equal to ") +
           (std::stringstream() << std::setprecision(12) << a).str())) {
  *result_listener << "( displayed with high precision: "
                   << std::setprecision(12) << arg << " )";

  return a == arg;
}

TEST(FloatComparison, UseCustomMatcher) {
  float expectedResult = 0.07263389974832534800;
  float actualResult = 0.07263390719890594500;

  EXPECT_THAT(expectedResult, IsFloatEq(actualResult));
  std::cout << "----------------------------------------------------"
            << std::endl;
}

Live example: https://godbolt.org/z/GhT3EaP1c

Output:

Gtest main is printing[==========] Running 4 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 4 tests from FloatComparison
[ RUN      ] FloatComparison.UsePred
example.cpp:19: Failure
Value of: MyFloatEq(expectedResult, actualResult)
  Actual: false (because 0.072633899748325348 != 0.072633907198905945)
Expected: true
----------------------------------------------------
[  FAILED  ] FloatComparison.UsePred (0 ms)
[ RUN      ] FloatComparison.UsePredFormat
example.cpp:39: Failure
Expected equality of these values:
  expectedResult
    Which is: 0.072633899748325348
  actualResult
    Which is: 0.072633907198905945

----------------------------------------------------
[  FAILED  ] FloatComparison.UsePredFormat (0 ms)
[ RUN      ] FloatComparison.PrintExtraInfo
example.cpp:59: Failure
Expected equality of these values:
  expectedResult
    Which is: 0.0726339
  actualResult
    Which is: 0.0726339
( displayed with high precision: 0.0726338997483, 0.0726339071989 )
----------------------------------------------------
[  FAILED  ] FloatComparison.PrintExtraInfo (0 ms)
[ RUN      ] FloatComparison.UseCustomMatcher
example.cpp:82: Failure
Value of: expectedResult
Expected: should be equal to 0.0726339071989
  Actual: 0.0726339 (of type float), ( displayed with high precision: 0.0726338997483 )
----------------------------------------------------
[  FAILED  ] FloatComparison.UseCustomMatcher (0 ms)
user_13833560

This is the exact code I ended up using, based on Ari's method #2 in the accepted answer:

/* Implement a CUSTOM_EXPECT_EQ macro instead of GoogleTest's standard EXPECT_EQ in order to print
 * more digits for floating point failures.
 * E.g. For this EXPECT_EQ(expectedResult, m_measData.voltage[ePhaseA]) failure...
 *              Expected equality of these values:
 *                expectedResult
 *                  Which is: 0.0726339
 *                m_measData.voltage[ePhaseA]
 *                  Which is: 0.0726339
 * ...(note that expected & actual are rounded and appear the same!), with CUSTOM_EXPECT_EQ,
 * we would see this instead:
 *              Expected equality of these values:
 *                expectedResult
 *                  Which is: 0.072633899748325348
 *                m_measData.voltage[ePhaseA]
 *                  Which is: 0.072633907198905945
 */
#define CUSTOM_EXPECT_EQ(expected,actual) EXPECT_PRED_FORMAT2(floatEqPred, expected, actual)
testing::AssertionResult floatEqPred(char const* expectedFmt, char const* actualFmt, float32_t expected, float32_t actual)
{
   return (expected == actual)
      ? testing::AssertionSuccess()
      : testing::AssertionFailure() << "Expected equality of these values:\n  "
              << expectedFmt << "\n    Which is: " << expected << "\n  "
              << actualFmt << "\n    Which is: " << actual << "\n";
}

The main thing I added to Ari's solution is that I wrapped it in a macro named CUSTOM_EXPECT_EQ in order to make it easier to use.

Incidentally I found that there were two causes of the intermittent failure I was seeing:

  • Someone added a unittest that, due to order-of-operations and a missing set of parentheses in the test, didn't compute the expectedResult in exactly the same way as the production code. It worked most of the time but failed occasionally for some inputs.

  • Even with the order-of-operations issue fixed (by adding parentheses), the test was still failing for some combination of inputs, which I finally tracked down to a GCC "bug" that has been reported & rejected many, many times! See GCC Bug 323 - optimized code gives strange floating point results. Since our test code doesn't need to operate on older CPUs, I added these compiler flags to our GCC command line: -msse2 -mfpmath=sse and now finally the test runs for all inputs. (Well, the inputs are pseudorandom, so maybe I haven't seen all 180 quintillion input combinations, but I haven't seen it fail in many, many runs.)

user_12454200
Related