How to get reproducible result when running Keras with Tensorflow backend

user_4544561 0
Viewed 11498

Every time I run LSTM network with Keras in jupyter notebook, I got a different result, and I have googled a lot, and I have tried some different solutions, but none of they are work, here are some solutions I tried:

  1. set numpy random seed

    random_seed=2017 from numpy.random import seed seed(random_seed)

  2. set tensorflow random seed

    from tensorflow import set_random_seed set_random_seed(random_seed)

  3. set build-in random seed

    import random random.seed(random_seed)

  4. set PYTHONHASHSEED

    import os os.environ['PYTHONHASHSEED'] = '0'

  5. add PYTHONHASHSEED in jupyter notebook kernel.json

    { "language": "python", "display_name": "Python 3", "env": {"PYTHONHASHSEED": "0"}, "argv": [ "python", "-m", "ipykernel_launcher", "-f", "{connection_file}" ] }

and the version of my env is:

Keras: 2.0.6
Tensorflow: 1.2.1
CPU or GPU: CPU

and this is my code:

model = Sequential()
model.add(LSTM(16, input_shape=(time_steps,nb_features), return_sequences=True))
model.add(LSTM(16, input_shape=(time_steps,nb_features), return_sequences=False))
model.add(Dense(8,activation='relu'))        
model.add(Dense(1,activation='linear'))
model.compile(loss='mse',optimizer='adam')
lstm keras tensorflow deep-learning
3 Answers

Keras + Tensorflow.

Step 1, disable GPU.

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ""

Step 2, seed those libraries which are included in your code, say "tensorflow, numpy, random".

import tensorflow as tf
import numpy as np
import random as rn

sd = 1 # Here sd means seed.
np.random.seed(sd)
rn.seed(sd)
os.environ['PYTHONHASHSEED']=str(sd)

from keras import backend as K
config = tf.ConfigProto(intra_op_parallelism_threads=1,inter_op_parallelism_threads=1)
tf.set_random_seed(sd)
sess = tf.Session(graph=tf.get_default_graph(), config=config)
K.set_session(sess)

Make sure these two pieces of code are included at the start of your code, then the result will be reproducible.

user_95013910

I resolved this issue by adding os.environ['TF_DETERMINISTIC_OPS'] = '1'

Here an example:

import os
os.environ['TF_DETERMINISTIC_OPS'] = '1'
#rest of the code
#TensorFlow version 2.3.1
user_89432140
Related