precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Check if root of cubic equation is complex or not?
user_46084910
•
2
1
77
math.net complex-numbers precision floating-point c#
Do we need epsilon value for lesser or greater comparison for float value?
user_42538340
•
2
5
275
precision floating-point optimization algorithm c++
How to perform binomial-coefficient and factorial calculation with more precision?
user_20912720
•
1
1
96
binomial-coefficients wolfram-mathematica precision numpy python
Do FMA instructions of different CPUs have different intermediate accuracy? If yes, then how does a compiler equalize the floating-point behavior?
user_14700920
•
4
1
93
floating-accuracy precision floating-point gcc c++
compute probabilities from log scores
user_117632330
•
1
1
31
probability precision r rounding
Strange warning when calling sprintf with .* width specifier
user_68659320
•
3
3
132
buffer-overflow precision printf double c
sprintf precision typecasting leading to compilation warning
user_141747710
•
2
2
158
precision printf integer c++
Check if number should be C++ float or C++ double using python
user_108545610
•
2
0
75
narrowing precision python c++
Overwrite toFixed() with appropriate replacement to fix floating point error javascript
user_27434860
•
2
2
130
tofixed precision floating-point javascript
What is the minimum number of significant decimal digits in a floating point literal to represent the value as correct as possible?
user_176658070
•
6
3
307
numerical-methods ieee-754 precision floating-point c
Hot Questions