Capturing epoch count when using EarlyStopping feature with Keras Model

user_16985121 0
Viewed 192

I working my way through using Keras Modeling and think I have now sussed how to use the callback feature to trap the best fit and prevent overfitting; all seems good. Whilst I can understand the verbose parameter will display the information I require it makes the output messy and I prefer to set this to zero. I would though still like to somehow capture the "epoch" count that gave the best result to incorporate into my own display; is there some way I can get at this? Thanks

    model.compile(optimizer='adam', loss='mse' )] 
    cbfile = 'best_model.h5'
    calls = [
    EarlyStopping(monitor='val_loss', mode='auto', verbose=0, patience=10),\
    ModelCheckpoint(cbfile, monitor = 'val_loss', mode = 'auto',\
            save_best_only = True ) ]
    history = model.fit(Xvect, Yvect, epochs=mcycl, batch_size=32,\
            validation_split=dsplit, verbose=0, callbacks = calls )
    saved = load_model('best_model.h5')        
    score = saved.evaluate(Xvect, Yvect, verbose=0)
    print('"Overall loss for best fit":',np.round(score,4))
keras tensorflow callback model python
1 Answers

How about writing your own custom EarlyStopping callback? The Tensorflow docs provide a very good example of how you could get started:

import numpy as np


class EarlyStoppingAtMinLoss(keras.callbacks.Callback):
    """Stop training when the loss is at its min, i.e. the loss stops decreasing.

  Arguments:
      patience: Number of epochs to wait after min has been hit. After this
      number of no improvement, training stops.
  """

    def __init__(self, patience=0):
        super(EarlyStoppingAtMinLoss, self).__init__()
        self.patience = patience
        # best_weights to store the weights at which the minimum loss occurs.
        self.best_weights = None

    def on_train_begin(self, logs=None):
        # The number of epoch it has waited when loss is no longer minimum.
        self.wait = 0
        # The epoch the training stops at.
        self.stopped_epoch = 0
        # Initialize the best as infinity.
        self.best = np.Inf

    def on_epoch_end(self, epoch, logs=None):
        current = logs.get("loss")
        if np.less(current, self.best):
            self.best = current
            self.wait = 0
            # Record the best weights if current results is better (less).
            self.best_weights = self.model.get_weights()
        else:
            self.wait += 1
            if self.wait >= self.patience:
                self.stopped_epoch = epoch
                self.model.stop_training = True
                print("Restoring model weights from the end of the best epoch.")
                self.model.set_weights(self.best_weights)

    def on_train_end(self, logs=None):
        if self.stopped_epoch > 0:
            print("Epoch %05d: early stopping" % (self.stopped_epoch + 1))

Note the self.stopped_epoch variable in the example. This way you have full control of what you display and how your early stopping logic works. Furthermore, using the logs dictionary, you can access your current loss and accuracy for epoch x. On the other hand, if you just want to use a simple print statement after training your model, you could just get the last epoch of your callback and print it:

model.compile(optimizer='adam', loss='mse' )] 
cbfile = 'best_model.h5'
early_stopping = EarlyStopping(monitor='val_loss', mode='auto', verbose=0, patience=10)

calls = [early_stopping,
ModelCheckpoint(cbfile, monitor = 'val_loss', mode = 'auto',\
            save_best_only = True ) ]
history = model.fit(Xvect, Yvect, epochs=mcycl, batch_size=32,\
            validation_split=dsplit, verbose=0, callbacks = calls )
saved = load_model('best_model.h5')        
score = saved.evaluate(Xvect, Yvect, verbose=0)

print('"Overall loss for best fit":',np.round(score,4)) 
print("Epoch %05d: early stopping" % (early_stopping.stopped_epoch + 1))
user_96578610
Related