effect of floating point precision in testing if a varable attains a bound

Question

I have a C/C++ function checkt (FLT t) where FLT is the floating point precison set by the preprocessor that I want to use in a much bigger body of code to ensure that values passed to it are in the range [0,1]. It is often used in increasing a variable by fixed increments checkt trips an error when the argument is out of range. In this simple example I want to go from 0.0 to 1.0 in ten increments of 0.1. However since 0.1 is not represented perfectly as 0.1000000000 etc to the limit of precision and then I will attain exactly the value 1.0 at the end of the loop. The code snippet below works if FLT is set to double but not if it is set to float. I also append the output from both. The compiler is gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0.

#include 
#include 

#define FLT double

void checkt (FLT t) {
    if (t < FLT{0} || t > FLT{1}) {
        std::cout << "t before error = " << t << std::endl;
        throw std::runtime_error ("t out of range [0,1]");
    }
}

int main(int argc, char **argv) {
    FLT x = FLT{0.0};
    FLT inc = FLT{0.1};
    FLT y = FLT{1};

    std::cout.precision(16);
    std::cout << "x is " << x << "y is " << y << " inc is " << inc << std::endl;

    for (int i=0; i<11; i++){ 
        std::cout.precision(16);
        std::cout << "x is " << x << std::endl;
        checkt(x);
        x += inc;
    }

    return 0;
}

Output with FLT=double x is 0y is 1 inc is 0.1

x is 0
x is 0.1
x is 0.2
x is 0.3
x is 0.4
x is 0.5
x is 0.6
x is 0.7
x is 0.7999999999999999
x is 0.8999999999999999
x is 0.9999999999999999

output with FLT = float is

y is 1 inc is 0.1000000014901161
x is 0
x is 0.1000000014901161
x is 0.2000000029802322
x is 0.300000011920929
x is 0.4000000059604645
x is 0.5
x is 0.6000000238418579
x is 0.7000000476837158
x is 0.8000000715255737
x is 0.9000000953674316
x is 1.00000011920929
t before error = 1.00000011920929
terminate called after throwing an inst

user_1687119 · Answer

Because of the imprecise nature of binary floating point, the repeated addition in each iteration of the loop can result in an error factor accumulating over time.

Rather than adding the increment amount to x on each iteration, multiply the increment by the loop index and set x to that.

for (int i=0; i<11; i++){
    x = inc * i;
    std::cout.precision(17);
    std::cout << "x is " << x << std::endl;
    checkt(x);
}

Output for double:

x is 0y is 1 inc is 0.1
x is 0
x is 0.10000000000000001
x is 0.20000000000000001
x is 0.30000000000000004
x is 0.40000000000000002
x is 0.5
x is 0.60000000000000009
x is 0.70000000000000007
x is 0.80000000000000004
x is 0.90000000000000002
x is 1

Output for float:

x is 0y is 1 inc is 0.1000000014901161
x is 0
x is 0.10000000149011612
x is 0.20000000298023224
x is 0.30000001192092896
x is 0.40000000596046448
x is 0.5
x is 0.60000002384185791
x is 0.69999998807907104
x is 0.80000001192092896
x is 0.90000003576278687
x is 1

effect of floating point precision in testing if a varable attains a bound

1 Answers