Why does the accuracy drop to zero in each epoch, while training convlstm layers in keras?

user_8895525 0
Viewed 1140

I am trying to use ConvLSTM layers in Keras 2 to train an action recognition model. The model has 3 ConvLSTM layers and 2 Fully Connected ones.

At each and every epoch the accuracy for the first batch (usually more than one) is zero and then it increases to some amount more than the previous epoch. For example, the first epoch finishes at 0.3 and the next would finish at 0.4 and so on.

My question is why does it get back to zero at each epoch?

p.s.

  • The ConvLSTM is stateless.
  • The model is compiled with SGD(lr=0.001, decay=1e-6, momentum=0.9, nesterov=True), for some reason it does not converge using Adam.
lstm conv-neural-network keras tensorflow deep-learning
1 Answers

So - in order to understand why something like this happening you need to understand how keras computes accuracy during the batch computation:

  1. Before each batch - a number of positively classified examples is stored.
  2. After each batch - a number of positively classified examples is stored and it's printed after division by all examples used in training.

As your accuracy is pretty low it's highly probable that in a first few batches none of the examples will be classified properly. Especially when you have a small batch. This makes accuracy to be 0 at the beginning of your training.

user_59744330
Related