tf.keras model.predict each time provides different values

user_2903597 0
Viewed 799

Each time I run:

y_true = np.argmax(tf.concat([y for x, y in train_ds], axis=0), axis=1)
y_pred = np.argmax(model.predict(train_ds), axis=1)

confusion_matrix(y_true, y_pred)

The result each time is different to my understanding the line: y_pred = np.argmax(model.predict(train_ds), axis=1) is different each time.

Clarification: I run cell 1 (training) once. And cell 2 (inference) few times.

Why?

THE CODE: Cell 1 (jupyter)

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Input, Conv2D, experimental
from tensorflow.keras.layers import MaxPool2D, Flatten, Dense 
from tensorflow.keras import Model
from tensorflow.keras.losses import categorical_crossentropy
from sklearn.metrics import accuracy_score


image_size = (100, 100)
batch_size = 32

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
    directory,
    label_mode='categorical',
    validation_split=0.2,
    subset="training",
    seed=1337,
    color_mode="grayscale",
    image_size=image_size,
    batch_size=batch_size,
)


inputs = Input(shape =(100,100,1))
x = experimental.preprocessing.Rescaling(1./255)(inputs)
x = Conv2D (filters =4, kernel_size =3, padding ='same', activation='relu')(x)
x = Conv2D (filters =4, kernel_size =3, padding ='same', activation='relu')(x)
x = MaxPool2D(pool_size =2, strides =2, padding ='same')(x)

x = Conv2D (filters =8, kernel_size =3, padding ='same', activation='relu')(x)
x = Conv2D (filters =8, kernel_size =3, padding ='same', activation='relu')(x)
x = MaxPool2D(pool_size =2, strides =2, padding ='same')(x)

x = Flatten()(x)
x = Dense(units = 4, activation ='relu')(x)
x = Dense(units = 4, activation ='relu')(x)
output = Dense(units = 5, activation ='softmax')(x)

model = Model (inputs=inputs, outputs =output)

model.compile(
    optimizer=tf.keras.optimizers.Adam(1e-3),
    loss=categorical_crossentropy,
    metrics=["accuracy"])

model.fit(train_ds, epochs=5)

Cell 2:

print (Accuracy:)
y_pred = np.argmax(model.predict(train_ds), axis=1)
print (accuracy_score(y_true, y_pred))

y_pred = np.argmax(model.predict(train_ds), axis=1)
print (accuracy_score(y_true, y_pred))

OUTPUT:

118/118 [==============================] - 7s 57ms/step - loss: 0.1888 - accuracy: 0.9398

Accuracy:

0.593

0.586

tf.keras keras tensorflow
2 Answers

Are you sure you do not train the model again every time you run the code? If the parameters of the model are the same the predicted result for the same input should be the same every time.

user_155196290

To my current understanding the reason of an above is the:

tf.keras.preprocessing.image_dataset_from_directory

While instance of it is:

type(train_ds)

tensorflow.python.data.ops.dataset_ops.BatchDataset

Reproduction:

First run:

[x for x, y in train_ds]

Output:

[<tf.Tensor: shape=(32, 100, 100, 1), dtype=float32, numpy=  array([[[[157.],
          [155.],
          [159.],

Second run:

[x for x, y in train_ds]

Output:

[<tf.Tensor: shape=(32, 100, 100, 1), dtype=float32, numpy=  array([[[[ 34.],
          [ 36.],
          [ 39.],
          ...,

The possible solution


imgs, y_true = [], []
for img, label in train_ds:
    imgs.append(img)
    y_true.append(label)

imgs = tf.concat(imgs, axis=0)    
y_true = np.argmax(tf.concat(y_true, axis=0), axis=1)

y_pred = np.argmax(model.predict(imgs), axis=1)
print (accuracy_score(y_true, y_pred))

y_pred = np.argmax(model.predict(imgs), axis=1)
print (accuracy_score(y_true, y_pred))

OUTPUT

0.944044764
0.944044764

Is there any better solution?

UPDATE 2:

Maybe more appropriate apporach in case of validation dataset (here the train_ds is just for example is to add an argument Shuffle=False)


    train_ds = tf.keras.preprocessing.image_dataset_from_directory(
        directory,
        label_mode='categorical',
        validation_split=0.2,
        subset="training",
        seed=1337,
        color_mode="grayscale",
        image_size=image_size,
        batch_size=batch_size,
        Shuffle=False

    )

UPDATE 3:

Here it's probably the best option in case if your test images are in a separate folder.

path = 'your path to test folder'


test_generator = ImageDataGenerator().flow_from_directory(
    directory=path,
    class_mode='categorical',
    shuffle=False,
    batch_size=32,
    target_size=(512, 512)
)

test_generator.reset()

This is better than OPTION 1, since it can work on dataset, which doesn't fits into memory (RAM).

user_29035970
Related