Reset weights to last epoch if loss value has increased Keras

user_10990546 0
Viewed 116

I am working on my ANN in Keras that is used with an imbalanced binary classification dataset and I have just set up a custom learning rate that checks at the start of each epoch what the loss value was compared to last epoch. If it's smaller I increase learning rate and if not I decrease learning rate and I want to reset the weights to the same as last epoch, how do I do this?

I have found something like

model.layers[0].get_weights()

Will this give me the weights? How can I then save them to my callback and set them if this condition is met?

class CustomLearningRateScheduler(keras.callbacks.Callback):

    def __init__(self):
        super(CustomLearningRateScheduler, self).__init__()
        self.lastVal = 0
        self.learning_rate = 10
        self.last_iteration_weights = []

    def on_train_begin(self, logs={}):
        self.errors = []

    def on_epoch_start(self, epoch):
        self.weights = self.model.layers[0].get_weights()

    def on_epoch_end(self, epoch, logs={}):
        if not hasattr(self.model.optimizer, "lr"):
            raise ValueError('Optimizer must have a "lr" attribute.')
        # Get the current learning rate from model's optimizer.
        lr = float(tf.keras.backend.get_value(self.model.optimizer.learning_rate))
 
        val = logs.get('loss')

        if(float(val) > float(self.lastVal)):
            self.learning_rate = lr * 0.95
            tf.keras.backend.set_value(self.model.optimizer.lr, self.learning_rate)
            
        else:
            self.learning_rate = lr * 1.01
            tf.keras.backend.set_value(self.model.optimizer.lr, self.learning_rate)
        self.lastVal = val
        self.errors.append(self.lastVal)

        print("\nEpoch %05d: Learning rate is %f ." % (epoch, self.learning_rate))

This class is called in:

model_p.fit(X, y, epochs=EPOCH_SIZE, batch_size=BATCH_SIZE, verbose=1, shuffle=True, callbacks=[CustomLearningRateScheduler()])
keras tensorflow machine-learning callback python
1 Answers

I have written a custom callback Dwell that achieves what you wish to accomplish and have used this callback on a large number of image classification tasks.

This callback enables you to select the option to continue or halt training. After training for ask_epoch number of epochs the callback queries the user to either enter H to halt traning or enter an integer N. If an integer is entered training will continue for N more epochs then the user is queried again. It also enables you to set a parameter called dwell. If dwell is set to True, the callback monitors validation loss. If at the end of an epoch the validation loss increases above the validation loss of the previous epoch , the model's weights are reset to the weights of the previous epoch and the learning rate is reduced by next_lr=current_lr * factor where factor is a user specified float value less than 1.0. The idea is that if validation loss increased the model has moved to a location in N space (N is the number of trainable weights) that is less favorable than the position in N space of the previous epoch. So why go there. Instead restore the weights of the previous epoch, then reduce the learning rate. The form of the callback is DWELL(model, factor, dwell, verbose, ask_epoch) where:

model is the name of your compiled model factor is a float between 0.0 and 1.0. If the validation loss increased the learning rate for the next epoch is determined by next_lr = current_lr * factor. dwell is a boolean. If set to True, validation loss is monitored. If it increased then the model weights are set to those of the previous epoch and the learning rate is reduced. verbose is a boolean. If True the callback prints out the new lr at the end of an epoch where the validation loss increased ask_epoch is an integer. At the start of training, training will take place for ask_epoch number of epochs. At that point the user is queried to enter H to halt training, or enter an integer N where N specifies how many more epochs to run than be queried again

class DWELL(keras.callbacks.Callback):
    def __init__(self,model,  factor,dwell, verbose,ask_epoch):
        super(DWELL, self).__init__()
        self.model=model
        self.initial_lr=float(tf.keras.backend.get_value(model.optimizer.lr)) # get the initiallearning rate and save it  
        self.lowest_vloss=np.inf # set lowest validation loss to infinity initially
        self.best_weights=self.model.get_weights() # set best weights to model's initial weights 
        self.verbose=verbose 
        self.best_epoch=0
        self.ask_epoch=ask_epoch
        self.ask=True
        self.dwell=dwell
    
    def on_train_begin(self, logs=None): # this runs on the beginning of training
        print('Training will proceed until epoch', ask_epoch,' then you will be asked to') 
        print('enter H to halt training or enter an integer for how many more epochs to run then be asked again')  
        self.start_time= time.time() # set the time at which training started
        
    
    def on_epoch_end(self, epoch, logs=None):  # method runs on the end of each epoch
        if self.ask: # are the conditions right to query the user?
            if epoch + 1 ==self.ask_epoch: # is this epoch the one for querying the user?
                print('\n Enter H to end training or  an integer for the number of additional epochs to run then ask again')
                ans=input()
                
                if ans == 'H' or ans =='h' or ans == '0': # quit training for these conditions
                    print ('you entered ', ans, ' Training halted on epoch ', epoch+1, ' due to user input\n', flush=True)
                    self.model.stop_training = True # halt training
                else: # user wants to continue training
                    self.ask_epoch += int(ans)
                    print ('you entered ', ans, ' Training will continue to epoch ', self.ask_epoch, flush=True)
        if self.dwell:
            lr=float(tf.keras.backend.get_value(self.model.optimizer.lr)) # get the current learning rate         
            vloss=logs.get('val_loss')  # get the validation loss for this epoch 
            if vloss>self.lowest_vloss:
                self.model.set_weights(self.best_weights)
                new_lr=lr * factor
                tf.keras.backend.set_value(self.model.optimizer.lr, new_lr)
                if self.verbose:
                    print( '\n model weights reset to best weights from epoch ', self.best_epoch+1, ' and reduced lr to ', new_lr, flush=True)
            else:
                self.lowest_vloss=vloss
                self.best_weights=self.model.get_weights()
                self.best_epoch= epoch

Here is an example of use

# model is the variable name of your compiled model
ask_epoch=5 # query user at end of epoch 5 to halt or continue training
factor=.5 # if validation loss increased next_lt = current_lr * factor
dwell=True
verbose=True  # print out new lr if validation loss increased
dwell=DWELL(model,factor, verbose,  ask_epoch)
callbacks=[ DWELL(model,factor,dwell, verbose,  ask_epoch)]

In model.fit set callbacks=callbacks. Below is an example of the training output where I purposefully set a large initial learning rate (.02) to cause the DWELL callback to reduce the learning rate fairly early in the training.

Training will proceed until epoch 5  then you will be asked to
enter H to halt training or enter an integer for how many more epochs to run then be asked again
Epoch 1/40
313/313 [==============================] - 62s 153ms/step - loss: 6.2284 - accuracy: 0.1456 - val_loss: 2.9476 - val_accuracy: 0.2458
Epoch 2/40
313/313 [==============================] - 44s 141ms/step - loss: 2.1466 - accuracy: 0.2686 - val_loss: 8.4516 - val_accuracy: 0.3502

 model weights reset to best weights from epoch  1  and reduced lr to  0.009999999776482582
Epoch 3/40
313/313 [==============================] - 46s 146ms/step - loss: 2.0746 - accuracy: 0.2628 - val_loss: 1.7664 - val_accuracy: 0.4072
Epoch 4/40
313/313 [==============================] - 45s 144ms/step - loss: 1.8257 - accuracy: 0.3944 - val_loss: 1.3599 - val_accuracy: 0.6120
Epoch 5/40
313/313 [==============================] - 45s 144ms/step - loss: 1.5230 - accuracy: 0.5530 - val_loss: 1.0913 - val_accuracy: 0.6901

 Enter H to end training or  an integer for the number of additional epochs to run then ask again
 2
you entered  2  Training will continue to epoch  7
Epoch 6/40
313/313 [==============================] - 44s 141ms/step - loss: 1.2793 - accuracy: 0.6745 - val_loss: 0.8224 - val_accuracy: 0.8284
Epoch 7/40
313/313 [==============================] - 45s 142ms/step - loss: 1.0747 - accuracy: 0.7442 - val_loss: 0.7990 - val_accuracy: 0.8271

 Enter H to end training or  an integer for the number of additional epochs to run then ask again
 4
you entered  4  Training will continue to epoch  11
Epoch 8/40
313/313 [==============================] - 45s 144ms/step - loss: 0.9850 - accuracy: 0.7770 - val_loss: 1.5557 - val_accuracy: 0.8688

 model weights reset to best weights from epoch  7  and reduced lr to  0.004999999888241291
Epoch 9/40
313/313 [==============================] - 45s 143ms/step - loss: 0.8708 - accuracy: 0.7911 - val_loss: 0.5515 - val_accuracy: 0.8643
Epoch 10/40
313/313 [==============================] - 45s 144ms/step - loss: 0.8346 - accuracy: 0.8047 - val_loss: 0.4961 - val_accuracy: 0.9129
Epoch 11/40
313/313 [==============================] - 45s 144ms/step - loss: 0.7811 - accuracy: 0.8364 - val_loss: 0.5186 - val_accuracy: 0.9526

 Enter H to end training or  an integer for the number of additional epochs to run then ask again
 h
you entered  h  Training halted on epoch  11  due to user input

I have made a number of test with dwell set to True and dwell set to False on the same data set. Because of inherent tensorflow randomness it is difficult to tell but it appears when dwell=True the model seems to converge a bit faster. I have had no issues to date with problems of convergence to local minimums with dwell=True I achieve as good or better validation loss than with dwell = False

user_107989170
Related