precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Difference in floating point precision behavior between C and C#
user_9280620
•
2
1
225
ieee-754 precision floating-point c c#
Why is the implementation of cross entropy different in Pytorch and Tensorflow?
user_15756480
•
5
2
649
pytorch tensorflow neural-network precision
How can I set floating point precision to avoid floating point arithmetic error in Python?
user_126487320
•
3
2
502
floating-accuracy precision floating-point python
Can std::isfinite(IntegralType) ever return false?
user_11504620
•
2
1
130
precision floating-point c++
Floating point types representation
user_134222430
•
2
2
117
ieee-754 precision floating-point memory c++
IEEE 754 conformant sqrt() implementation for double type
user_17782750
•
2
2
783
sqrt ieee-754 precision floating-point rounding
Difference in precision in np.float64 and python float (64)
user_30765440
•
2
1
101
precision printing numpy python
What is a robust method for calculating the cosine of the angle of two vectors in python?
user_32199420
•
2
2
137
trigonometry precision python-3.x numpy python
Precision is lost while reading a column from excel using openpyxl
user_82815090
•
2
1
286
openpyxl pandas precision python-3.x excel
glReadPixels from back buffer. Problem with float precision
user_50299560
•
2
1
164
precision floating-point opengl c++
Hot Questions