WHY tf.keras.layers.Conv2D gives different results at each run

user_18707675 0
Viewed 121

I am trying to reproduce my code from online Jupyter Notebook (COURSERA) to my own local environment （Anaconda 3 Jupyter with CUDA installed) All Codes are exactly same, and was working fine online

I imported Conv2D as usual:

import tensorflow as tf
import numpy as np
import scipy.misc
from tensorflow.keras.applications.resnet_v2 import ResNet50V2
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet_v2 import preprocess_input, decode_predictions
from tensorflow.keras import layers
from tensorflow.keras.layers import Input, Add, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D, AveragePooling2D, MaxPooling2D, GlobalMaxPooling2D
from tensorflow.keras.models import Model, load_model
from tensorflow.keras.initializers import random_uniform, glorot_uniform, constant, identity
from tensorflow.python.framework.ops import EagerTensor
from matplotlib.pyplot import imshow


%matplotlib inline

and fed

X1 = np.ones((1, 4, 4, 3)) * -1
X2 = np.ones((1, 4, 4, 3)) * 1
X3 = np.ones((1, 4, 4, 3)) * 3
X = np.concatenate((X1, X2, X3), axis = 0).astype(np.float32)
print(X.shape)
y = Conv2D(filters = 2, kernel_size = 1, strides = (2,2), padding = 'valid')(X)

print(y.numpy())

The output shape is always (3, 2, 2, 2), but the value changes every single run.

Environment: Ubuntu 21.10 TensorFlow: 2.8.0 NVIDIA-SMI 510.54 CUDA Version: 11.6

tf.keras keras tensorflow machine-learning python
1 Answers

It is because the kernel is randomly initialized everytime. Try setting a random seed and you should get deterministic results:

X1 = np.ones((1, 4, 4, 3)) 
X2 = np.ones((1, 4, 4, 3)) 
X3 = np.ones((1, 4, 4, 3))
X = np.concatenate((X1, X2, X3), axis = 0).astype(np.float32)
results = []
for _ in range(10):
  tf.random.set_seed(111)
  results.append(Conv2D(filters = 2, kernel_size = 1, strides = (2,2), padding = 'valid', kernel_initializer = glorot_uniform(seed=0))(X))

np.all(results == results[0])
# True ==> all the same

Also note what docs say regarding glorot_uniform(seed=0):

Note that seeded initializer will not produce same random values across multiple calls, but multiple initializers will produce same sequence when constructed with same seed value.

user_96578610
Related