Keras validation loss from model.fit() not expected

user_2992954 0
Viewed 2684

On training my network using model.fit() in Keras with loss='mean_squared_error' in model.compile():

model.compile(loss='mean_squared_error', optimizer=optimizers.Adam(decay=0.00001), metrics=['accuracy'])
hist=model.fit(X, Y, epochs=200, batch_size=2000,  verbose=2, shuffle="batch", validation_data=(Xval,Yval))

I get this at the end:

Epoch 200/200 23s - loss: 1.2060 - acc: 0.5168 - val_loss: 1.5213 - val_acc: 0.5294

Now if I manually check the loss on the validation set by

pre=model.predict(Xval)
print(np.mean(np.square(pre-Yval)))

I get 1.4587 and not 1.5213.

Why are they different? Why am I not getting 1.5213 as calculated by model.fit()? Am I missing something here? Doesn't model.fit() go through the entire validation set?

keras deep-learning
2 Answers

After analyzing this situation I think I know why you could be experiencing this. Judging by the output you get on your epoch 200, we can see that your model metrics at that moment were:

loss: 1.2060 - acc: 0.5168 - val_loss: 1.5213 - val_acc: 0.5294

This means that on that last epoch, the feed-forward model you are training obtained a loss of 1.2060 (and a validation one of 1.5213).

However, the training algorithm still has to back-propagate that final loss to update your model weights. Only after iterating over inputs, then calculating metrics, and finally back-propagate weights, is that an epoch is considered to "end". The same process continues for any additional epochs.

That is why you get different losses when you evaluate after that final epoch, as your model changed a bit after that last back-propagation updated your model, learning a bit more than on the previous step. This would also explain why you obtained a lower loss value after epoch 200, as your model had one more epoch of successful training.

user_39081700
Related