Runtime float calculation vs evaluated calculation

Viewed 90

My question is not about the fine points of floating-point calculations, but about the differences in how it's evaluated. Given this:

float a = 1f / 12;
int b = (int)(1f / a);

At runtime, b will be 12. But if I evaluate its expression in something like QuickWatch or LINQPad, it's 11. In fact, I'm unable to recreate the result of 11 at runtime using float, double, or decimal; truncation or Math.Floor.

What's behind the difference between dynamic evaluation of the expression and the evaluation at runtime?

EDIT:

Since I'm seeing comments about people getting a result different from mine, here is what I'm seeing.

The result at runtime:

Runtime result

The result when evaluated by VS QuickWatch:

Eval result

The result in LINQPad:

LINQPad result

My VS and framework versions:

VS and framework versions

1 Answers

I think it's a matter of floating point calculations

Program with 32Bit ( Any CPU -> prefer 32Bit checked )

enter image description here

Program with 64Bit

enter image description here

Visual Studio is wrapped in 32-bit, so I think that's because QuickWatch runs in 32-bit.

This is like calculating (int)(1f/12) in QuickWatch

enter image description here

So I think it would be better to check what was calculated

enter image description here

Related