High accuracy but bad predictions on Keras Tensorflow

user_11516090 0
Viewed 778

I have a 9 class dataset with 7000 images, I use MobilenetV2 for training my set and ImageGenerator, resulting in 82% percent val accuracy. But when i predict my test images, it always predicts a false class. I have no idea what is wrong with it.Here is my code;

My ImageGenerator:

image_gen = ImageDataGenerator(rotation_range = 20,
                               width_shift_range=0.12,
                               height_shift_range=0.12,
                               shear_range=0.1,
                               zoom_range = 0.06,
                               horizontal_flip=True,
                               fill_mode='nearest',
                               rescale=1./255)

My model:

Model = Sequential()

Model.add(Conv2D(filters=32,kernel_size=(3,3),input_shape=image_shape,activation='relu'))
Model.add(MaxPool2D(pool_size=(2,2)))

Model.add(Conv2D(filters=64,kernel_size=(3,3),input_shape=image_shape,activation='relu'))
Model.add(MaxPool2D(pool_size=(2,2)))

Model.add(Conv2D(filters=64,kernel_size=(3,3),input_shape=image_shape,activation='relu'))
Model.add(MaxPool2D(pool_size=(2,2)))

Model.add(Conv2D(filters=64,kernel_size=(3,3),input_shape=image_shape,activation='relu'))
Model.add(MaxPool2D(pool_size=(2,2)))

Model.add(Flatten())

Model.add(Dense(256,activation='relu'))

Model.add(Dense(9,activation='softmax'))

MobilenetV2:

height=224
width=224
img_shape=(height, width, 3)
dropout=.3
lr=.001
class_count=9 # number of classes
img_shape=(height, width, 3)
base_model=tf.keras.applications.MobileNetV2( include_top=False, input_shape=img_shape, pooling='max', weights='imagenet') 
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(512, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
                bias_regularizer=regularizers.l1(0.006) ,activation='relu', kernel_initializer= tf.keras.initializers.GlorotUniform(seed=123))(x)
x=Dropout(rate=dropout, seed=123)(x)        
output=Dense(class_count, activation='softmax',kernel_initializer=tf.keras.initializers.GlorotUniform(seed=123))(x)
Model = keras.models.Model(inputs=base_model.input, outputs=output)
Model.compile( loss='categorical_crossentropy', metrics=['accuracy'],optimizer='Adamax')

My Rlronp:

rlronp=tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=1, verbose=1, mode='auto', min_delta=0.0001, cooldown=0, min_lr=0)

My train_image_gen:

train_image_gen = image_gen.flow_from_directory(train_path,
                                                target_size=image_shape[:2],
                                                color_mode='rgb',
                                                batch_size=batch_size,
                                                class_mode='categorical')

My test_image_gen:

test_image_gen = image_gen.flow_from_directory(test_path,
                                                target_size=image_shape[:2],
                                                color_mode='rgb',
                                                batch_size=batch_size,
                                            class_mode='categorical',shuffle=False)

My earlystop:

early_stop = EarlyStopping(monitor='val_loss',patience=4)

My Model fit:

results = Model.fit(train_image_gen,epochs=20,
                              validation_data=test_image_gen,callbacks=[rlronp,early_stop],class_weight=class_weight
                              )

Training and accuracy:

Epoch 20/20 200/200 [==============================] - 529s 3s/step -
 loss: 0.3995 - accuracy: 0.9925 - val_loss: 0.8637 - val_accuracy: 0.8258

My problem is when i predict an image from test set, it predicts the false class, 90% of time.

For example here, it has to be 3rd class, but max is on 2nd class.

array([[0.08064549, 0.04599327, 0.27055973, 0.05219262, 0.055945  ,
        0.25723988, 0.07608379, 0.10404343, 0.05729679]], dtype=float32)

I tried collecting my own dataset with 156 class and 2.5k images, but it was even worse.

My loss on 20 epochs: loss

jupyter-notebook keras tensorflow machine-learning python
2 Answers

accuracy: 0.9925; val_accuracy: 0.8258

Clearly the model is overfitted,

  1. Try using regularization techniques such as L2,L1 or Dropout, they will work.
  2. Try to Collect More data(Or use data augumentation)
  3. Or search for other Neural Network Architectures
  4. The best method is plot val_loss v/s loss
r = model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=15)

import matplotlib.pyplot as plt
plt.plot(r.history['loss'], label='loss')
plt.plot(r.history['val_loss'], label='val_loss')
plt.legend()

Plot

and check the point where loss and val_loss meet each other and then at the point of intersection see the number of epochs (say x) and train the model for x epochs only.

Hope you will find this useful.

user_132868590

Model is overfitted...use dropout layer.. I think it will help

Model.add(Dropout(0.2))
user_131518260
Related