keras CNN : train and validation set are identical but with different accuracy

user_4560470 0
Viewed 1029

I know this a very bad thing to do but I noticed something strange using keras mobilenet :

I use the same data for training and validation set :

train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(IM_WIDTH, IM_HEIGHT),
    batch_size=batch_size,
    class_mode = "categorical"
)
validation_generator = train_datagen.flow_from_directory(
  train_dir,
  target_size=(IM_WIDTH, IM_HEIGHT),
  class_mode = "categorical"
)

but I don't get the same accuracy on both !

epoch 30/30 - loss: 0.3485 - acc: 0.8938 - val_loss: 1.7545 - val_acc: 0.4406

It seems that I am overfitting the training set compared to the validation set.. but they are supposed to be the same ! How is that possible ?

keras tensorflow deep-learning
2 Answers

The training loss is calculated on the fly and only the validation loss is calculated after the epoch is trained. So at the beginning a nearly untrained net will make the training loss worse that it actually is. This effect should vanish in later epochs, since then one epochs mpact on the scoring is not that big anymore.

This behaviour is adressed in keras faq. If you evaluate both at the end of epoch with a self written callback, they should be the same.

user_93054500

For people reading this after a while : I still don't understand how this issue happened but it helped a lot working on the batchsize (reducing it).

user_45604700
Related