precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to make spaces in zeroes position in cobol
user_137665630
•
1
1
144
cobol alphanumeric numeric move precision
How is machine epsilon related to floating point rounding error?
user_162244370
•
1
1
640
epsilon rounding-error precision floating-point
numpy.linspace Floating Point Inaccuracy
user_33836400
•
2
1
91
precision rounding numpy python
Why are the same 32-bit floats different in JavaScript and Rust?
user_133782470
•
4
2
339
rust precision floating-point javascript
gcc 9.3.0 printf %Lf output not correct
user_76964430
•
2
1
118
long-double precision floating-point printf gcc
Precision Rounding must be positive
user_160358940
•
2
1
373
odoo-13 odoo precision python
why does the number 51444.325061 doesnot round off to 51444.325 upto 3 decimal places? I want to round off this number in c programming
user_158954750
•
2
2
105
precision decimal floating-point rounding c
student cdf and normal ppf precision
user_31778990
•
2
0
114
scipy.stats python-3.7 scipy precision
Why C++ strtod parses "708530856168225829.3221614e9" to 7.08530856168225898e+26 instead of 7.08530856168225761e+26?
user_3207260
•
3
2
57
ieee-754 precision floating-point parsing
php precision with big numbers
user_33339110
•
2
1
239
precision apache2 php
Hot Questions