precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
C: Rounding floats
user_141044070
•
4
1
133
precision floating-point c
For loop in which the counter goes up to a value generated by Math.floor()
user_152536360
•
2
2
77
floating-accuracy precision math for-loop javascript
fread/fwrite introduces garbage values
user_105180660
•
3
1
92
data.table precision r c++
Is it ok that with fp:fast 3000.f/1000.f != 3.f?
user_17826850
•
5
1
97
visual-studio-2019 precision floating-point visual-c++ c++
double precision in C++, 308 digits or 15 digits?
user_150549960
•
3
2
154
precision double c++
How to emulate Excel's automatic rounding?
user_58039100
•
2
1
45
precision floating-point rounding excel
np.float32 floating point differences between intel MacBook and M1
user_28852800
•
5
1
293
float32 apple-m1 precision numpy
Numpy array float precision is not deterministic
user_105490440
•
2
1
226
precision floating-point numpy arrays python
How do resolve problem with rounding problem in JavaScript
user_155174400
•
1
0
39
angular precision javascript
Numerical evaluation of arctangent functions
user_23294680
•
1
1
59
atan atan2 precision matlab numpy
Hot Questions