precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Is floating-point math consistent in C#? Can it be?
user_2384190
•
159
10
16832
ieee-754 precision floating-point .net c#
Double precision floating values in Python?
user_4091020
•
116
5
398568
precision python types
Converting 8 bytes of little-endian binary into a double precision float
user_8001900
•
12
5
8924
endianness precision floating-point parsing java
How to use Gcc 4.6.0 libquadmath and __float128 on x86 and x86_64
user_1965610
•
17
1
15387
128-bit precision floating-point gcc
Is it possible to remove floating point errors without resorting to arbitrary-precision datatypes?
user_167840
•
10
8
4312
precision floating-point rounding
Auto-Interval precision in MS Chart
user_2919990
•
15
7
15102
mschart data-visualization intervals precision c#
PHP - Floating Number Precision
user_2969360
•
93
8
55717
floating-accuracy precision floating-point php
Is there a way to get the "significant figures" of a decimal?
user_1055700
•
17
3
4211
significant-digits precision decimal .net
Difference between toFixed() and toPrecision()?
user_4026200
•
152
9
67590
precision rounding javascript
How to Java String.format with a variable precision?
user_872070
•
31
4
29349
floating-accuracy precision rounding java string
Hot Questions