Access loss and model in a custom callback

user_14721684 0
Viewed 1615

I read this https://www.tensorflow.org/guide/keras/custom_callback, but I don't know how I could get all the other parameters.

This is my code

 (hits, ndcgs) = evaluate_model(model, testRatings, testNegatives, topK, evaluation_threads)
  hr, ndcg, loss = np.array(hits).mean(), np.array(ndcgs).mean(), hist.history['loss'][0]
  print('Iteration %d [%.1f s]: HR = %.4f, NDCG = %.4f, loss = %.4f [%.1f s]' 
                  % (epoch,  t2-t1, hr, ndcg, loss, time()-t2))
 if hr > best_hr:
     best_hr, best_ndcg, best_iter = hr, ndcg, epoch
 if args.out > 0:
     model.save(model_out_file, overwrite=True)

As you can see I need model, hist and model.save. Is there a way to use these three parameters in a custom callback? So that I can write all of this into a custom callback.

class CustomCallback(keras.callbacks.Callback):

   def on_epoch_end(self, logs=None):
       keys = list(logs.keys())
       print("Stop training; got log keys: {}".format(keys))
keras tensorflow callback python
1 Answers

The model is an attribute of tf.keras.callbacks.Callback, so you can access it directly with self.model. For accessing the value of the loss, you can use the "logs" object that is passed to the methods of tf.keras.callbacks.Callback, that will contain a key named "loss".

If you need to access to other variables (that won't change during the training), then you can set them as instance variables of your callback, and add them during the construction of the callback by defining the __init__ function.

class CustomCallback(keras.callbacks.Callback):
   def __init__(self, testRatings, testNegatives, topK, evaluation_threads):
       super().__init__()
       self.testRatings = testRatings
       self.testNegatives = testNegatives
       self.topK = topK
       self.evaluation_threads = evaluation_threads

   def on_epoch_end(self, epoch, logs=None):
       logs = logs or {}
       current_loss = logs.get("loss")
       if current_loss:
           print("my_loss: ", current_loss)
       print("my_model", self.model)
       # the attributes are accessble with self
       print("my topK atributes", self.topK)

# you can then create the callback by passing the correct attributes
my_callback = CustomCallback(testRatings, testNegatives, topK, evaluation_threads)

Note: If what you want to do is evaluate the model between each epoch, and save the model if the model got the best metrics, I suggest you give a look to:

  • The fit function, where you can actually provide a testing set
  • The metrics module, that provides metrics that will be computed on the train and the test set
  • The ModelCheckpoint callback, that will save the model at each epoch, keeping the best weights if providing the option save_best_only
user_73701530
Related