floating-point

961 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Numpy median precision issues at scale
user_14674340
•
2
1
172
median precision floating-point numpy python
How do I take the average of a large floating point array precisely?
user_112576280
•
3
3
585
avx simd precision floating-point assembly
On 32-bit machines, atan2 is nondeterministic when I don't store its result in a variable. Why?
user_58639870
•
16
0
243
ieee-754 32-bit floating-point c c++
Why does C#'s decimal use binary integer significand?
user_458430
•
3
1
113
128-bit decimal floating-point .net c#
Float/Double doesn't seem to be working. The result keeps coming out to be an integer
user_122102750
•
2
3
144
floating-point double java
Floating point compare of absolute values in AVX
user_1491380
•
4
1
592
avx simd floating-point x86 performance
I got this error on early stopping function TypeError: '>' not supported between instances of 'NoneType' and 'float' in python
user_128609240
•
2
0
123
conv-neural-network non-type deep-learning floating-point python
Question of an example for storing value of FLOAT type
user_116507280
•
2
1
74
sqldatatypes floating-point
Why is mpfr_printf different than printf for hex float (%a conversion specifier)?
user_122886370
•
2
2
187
mpfr floating-point c++
What does line is supposed to mean?
user_138881870
•
2
2
82
pointers floating-point c reference c++
Hot Questions