Tensorflow adding floats gives additional digits

user_10375277 0
Viewed 191

So I was wondering why I do not get the sum of the two floats below, when adding them. Depending on the length of the float and whether there is a non zero digit before the point, the result has either an added or substracted additional digit or they just do not get added. The code below for example:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

with tf.Graph().as_default():
    a = tf.Variable(1, name="Var_a")
    dx = tf.Variable(1.33333, name='dx')
    dt = tf.Variable(5e-10, name='dt')
    dz = tf.Variable(1.0, name='dz')


    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        a = dt+dx
        a = a-dz
        print dx.eval()
        print dt.eval()
        print (dt+dx).eval()
        print (dx+dt).eval()
        print a.eval()
        writer = tf.summary.FileWriter("/tmp/tensorlog")
        writer.add_graph(sess.graph)

gives the following result:

>>>1.33333
>>>5e-10
>>>1.33333
>>>1.33333
>>>0.33333004

So the addition did not happen and the subtraction does not give a right result either. Thanks in advance for any ideas on this.

tensorflow precision python
1 Answers

TensorFlow variables are 32 bit floats by default, and they do not have enough precision to track the difference dt, which is too small in relation to the rest of values. Use dtype=tf.float64 to increase the precision and you will see the correct results:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

with tf.Graph().as_default():
    a = tf.Variable(1, name="Var_a", dtype=tf.float64)
    dx = tf.Variable(1.33333, name='dx', dtype=tf.float64)
    dt = tf.Variable(5e-10, name='dt', dtype=tf.float64)
    dz = tf.Variable(1.0, name='dz', dtype=tf.float64)


    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        a = dt+dx
        a = a-dz
        print dx.eval()
        print dt.eval()
        print (dt+dx).eval()
        print (dx+dt).eval()
        print a.eval()
        writer = tf.summary.FileWriter("/tmp/tensorlog")
        writer.add_graph(sess.graph)

Output:

1.33333
5e-10
1.3333300005
1.3333300005
0.33333000049999995

Note however that using tf.float64 instead of tf.float32 values comes at a cost in terms of memory and time, so use tf.float64 only when you really require high precision. In general, 32-bit float values are precise to about 7 decimal places while 64-bit float values are precise to about 15 decimal places.

user_17827920
Related