I am working on an LSTM based model to predict logs-anomaly. My model architecture is as given:
______________________Layer (type) Output Shape Param # =================================================================
lstm (LSTM) (None, 5, 1555) 9684540 ______________________
lstm_1 (LSTM) (None, 1555) 19350420 ______________________
dense (Dense) (None, 1024) 1593344 ______________________
dense_1 (Dense) (None, 1024) 1049600 _______________________
dense_2 (Dense) (None, 1555) 1593875
=================================================================
Total params: 33,271,779
I want to go for continual training avoiding catastrophic forgetting, I saw this paper on EWC. Yes, I am going to get totally different log files on incremental training, so catastrophic forgetting is happenning currently. I looked on internet and found only pytorch and tensorflow implementation of it. I am not very fluent at them, I am looking for some tensorflow-2/keras implementation of the same. I do understand the paper but not how to implement it. Is it possible to do in keras, if yes how? Any other simple continual learning approach is welcome!