Accuracy Decreasing with higher epochs

user_5796133 0
Viewed 16686

I am a newbie to Keras and machine learning in general. I'm trying to build a binary classification model using the Sequential model. After some experimenting, I saw that on multiple runs(not always) I was getting a an accuracy of even 97% on my validation data in the second or third epoch itself but this dramatically decreased to as much as 12%. What is the reason behind this ? How do I fine tune my model ? Here's my code - 

model = Sequential()
model.add(Flatten(input_shape=(6,size)))
model.add(Dense(6,activation='relu'))
model.add(Dropout(0.35))
model.add(Dense(3,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['binary_accuracy'])
model.fit(x, y,epochs=60,batch_size=40,validation_split=0.2)
keras deep-learning classification machine-learning python
3 Answers

According to me, you can take into consideration the following factors.

  1. Reduce your learning rate to a very small number like 0.001 or even 0.0001.
  2. Provide more data.
  3. Set Dropout rates to a number like 0.2. Keep them uniform across the network.
  4. Try decreasing the batch size.
  5. Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.

If any of the above factors work for you, please let me know about it in the comments section.

Admin1

When I have this issue, I solved by changing the optimizer from RMSprop to adam. I also reduced the learning rate and added dropout after each fully connected layers. If your FC layers have small number of neurons then adding dropout does not make a major difference.

user_130858810

Reduce the learning rate (like 0.001 or 0.0001) and add batch normalization after every convolution layer.

user_157749820
Related