precision

318 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Single precision argument reduction for trigonometric functions in C
user_142871780
•
5
2
693
single-precision approximation precision floating-point c
What is the max value that Math.random can produce?
user_91191860
•
4
2
553
precision floating-point javascript
Reading bigint (int8) column data from Redshift without Scientific Notation using Pandas
user_131352150
•
5
2
643
amazon-redshift pandas precision numpy python
Why there are orders of magnitude in the evaluation of torch.sin() and numpy.sin() at integer multiples of Pi?
user_142840110
•
2
2
301
tensor pytorch precision numpy python
Haskell/GHC - Is there a way around Haskell only returning 16 decimal places?
user_45435600
•
3
1
114
eulers-number arbitrary-precision precision ghc haskell
Top k precision
user_63917030
•
3
1
1137
precision-recall information-retrieval precision
What is the machine precision in pytorch and when should one use doubles?
user_16015800
•
5
1
3260
pytorch precision floating-point
What operations can make floats leave a [0, 1] range?
user_57404280
•
7
2
162
ieee-754 precision floating-point c++
When is a floating point operation 'invalid'?
user_114490
•
2
1
972
numeric precision boost c++
Is the addition property of equality valid for floating point numbers?
user_23412850
•
2
2
58
algebra precision floating-point
Hot Questions