precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Creating a custom float class by inheritance in python
user_145306120
•
2
1
554
precision floating-point python-3.x constructor python
Largest floating point value that can be squared
user_44384630
•
2
1
111
numeric-limits precision floating-point c++
Floating points : Is it true if a > b then a - b > 0?
user_44933690
•
2
2
85
precision floating-point c++
Using extended precision in numpy prod
user_56726130
•
2
0
217
precision numpy python
Is floating point math more precise for values close to unity?
user_40561810
•
3
1
430
physics precision floating-point double
Why SQL Server cut precision to 3 or 6 places
user_57410560
•
2
1
90
precision sql-server sql
Does 10m * 0.1m equal 1m in C#?
user_414100
•
3
1
338
precision decimal c#
Numerical problems with qnorm
user_115181330
•
4
1
104
norm numeric precision r
std::fmod abysmal double precision
user_4853430
•
26
3
1675
fmod precision floating-point c++
ValueError: cross_val_predict only works for partitions
user_124190550
•
2
0
607
cross-validation precision python
Hot Questions