Tensorflow Callback: how to save best model on the memory not on the disk

user_7241236 0
Viewed 4977

I use Tensorflow for regression using the following function

import tensorflow as tf

def ff(*args, **kwargs):
    model = tf.keras.models.Sequential()
    model.add(tf.keras.Input(shape=[inp_train.shape[-1],]))
    for i in range(n_layer):
        model.add(tf.keras.layers.Dense(n_unit, activation=act))
    model.add(tf.keras.layers.Dense(out_train.shape[1]))
    model.compile(optimizer=opt, loss='mae')
    early_stop  = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=100)
    check_point = tf.keras.callbacks.ModelCheckpoint('best_model.h5', monitor='val_loss', save_best_only=True)
    model.fit(inp_train, out_train, epochs=n_epoch, batch_size=s_batch, validation_data=(inp_val, out_val), callbacks=[early_stop, check_point], verbose=0)
    best_model = tf.keras.models.load_model('best_model.h5')
    return model, best_mode

As you see, I save the best model by check_point callback and use it later for prediction. The problem is that in this way I have to save the best model on the disk first, and then load it from the disk. If I want to do a couple of runs in parallel, since each run create a file with the same name it does not work.

So, how can I assign the best model in a variable without having to save it on the disk?

tensorflow callback python
2 Answers

NOTE: I fixed a bug and is untested

I had to do this for myself and thought I would share:

Callback:

class SaveBestModel(tf.keras.callbacks.Callback):
    def __init__(self, save_best_metric='val_loss', this_max=False):
        self.save_best_metric = save_best_metric
        self.max = this_max
        if this_max:
            self.best = float('-inf')
        else:
            self.best = float('inf')

    def on_epoch_end(self, epoch, logs=None):
        metric_value = logs[self.save_best_metric]
        if self.max:
            if metric_value > self.best:
                self.best = metric_value
                self.best_weights = self.model.get_weights()

        else:
            if metric_value < self.best:
                self.best = metric_value
                self.best_weights= self.model.get_weights()

usage:

save_best_model = SaveBestModel()
model.fit(data, callbacks=[save_best_model]
#set best weigts
model.set_weights(save_best_model.best_weights)
user_26460870

Here is a basic example of creating a callback and saving the model at the time of callback to an external list. It has to be a list (or a type that allows modification using a method). The base tf.keras.callbacks.Callback class is extended with an additional argument, the list, in the callback class __init___ method. This example shows that it works. When the callback is called on training_end it appends the current model to the list.

import tensorflow as tf
from tensorflow.python.keras.models import Model

# define a custom callback
class MyCustomCallback(tf.keras.callbacks.Callback):

  def __init__(self, external_list):
      self.list_obj = external_list

  def on_train_end(self, logs=None):
      self.list_obj.append(self.model)

# test the idea works
model_save_list = []
my_callback = MyCustomCallback(model_save_list)

model1 = Model()
my_callback.set_model(model1)
my_callback.on_train_end()

print(model_save_list)

Run this and you will see the internal model gets added to your list object:

[<tensorflow.python.keras.engine.training.Model object at 0x10d230b50>]

Modify your training by adding your new callback to the callbacks like so:

model.fit(inp_train, out_train, epochs=n_epoch, batch_size=s_batch, validation_data=(inp_val, out_val), callbacks=[early_stop, my_callback], verbose=0)
user_54441730
Related