How to fit using float64?

user_2364295 0
Viewed 237

I've set all the layers in my model to use float64, yet when fitting the loss appears to still be coming out as float32 (based on the rounding I see). I'd like to ensure that all processing under the hood is in double. How can I guarantee that?

tensorflow2.0 tensorflow
1 Answers

instead of manually setting the dtype for each layer, you can define a global policy with:

policy = tf.keras.mixed_precision.Policy("float64")
tf.keras.mixed_precision.set_global_policy(policy)

or alternativly set the backend default float:

tf.keras.backend.set_floatx(policyConfig)

In that way all layers automatically use float64.

For your loss calculation: if intitialize values you should initialize them using tf.constant(0.0, dtype=tf.float64) instead of only initializing with 0.0 to ensure float64.

user_179057640
Related