precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Print floating point numbers with variable precision
user_125188410
•
3
2
190
precision floating-point python-3.x python
Correctly rounded computation of sqrt of sum of two floats handling overflow
user_18148810
•
2
3
267
numerical-methods precision floating-point language-agnostic
Where in the IEEE 754 standard is the result of `Inf * 0` specified?
user_32511140
•
2
1
97
numeric ieee-754 precision floating-point
Round-off to zero behavior in Maxima float coefficients
user_52973760
•
1
3
180
wxmaxima maxima precision
Is it possible to set the precision globally for an "f" string?
user_8279270
•
3
0
111
precision rounding python
What does the precision of float, double or long double mean in C++?
user_148256640
•
2
3
205
precision floating-point c++
effect of floating point precision in testing if a varable attains a bound
user_105915050
•
2
1
53
precision testing loops c++
tensorflow - how to use 16 bit precision float
user_42813530
•
2
1
2449
keras tensorflow precision python
How to control the display precision of a NumPy float64 scalar?
user_127278010
•
2
1
407
precision ipython format numpy
Is torch.float32 different from numpy's float32?
user_154441400
•
1
1
932
pytorch precision floating-point numpy python
Hot Questions