Keras - manage history

user_4841832 0
Viewed 811

I am training Keras models, saving them with model.save() and than later loading them and resuming training.

I would like to plot after each training the whole training history, but model.fit_generator() only returns the history of the last session of training.

I can save the history of the initial session and update it myself, but I wonder if there is a standard way in Keras of managing the training history.

history1 = model.fit_generator(my_gen)
plot_history(history1)
model.save('my_model.h5')

# Some days afterwards...

model = load_model('my_model.h5')
history2 = model.fit_generator(my_gen)

# here I would like to reconstruct the full_training history
# including the info from history1 and history2
full_history = ???
resuming-training keras tensorflow python
2 Answers

Let's say this line

print(history.history.keys())

produces the following output:

['acc', 'loss', 'val_acc', 'val_loss']

Based on the assumption that the loaded model should have the same performance as saved model, you can try something like concatenating histories. For example, concatenate new accuracy history on the loaded accuracy history of the loaded model.

It should start from the same point in the plotting space where the loaded model ended (maybe you will have to add (+) epochs of the previously trained model for the plot so that new values of accuracy don't start from epoch 0, but loaded models' last epoch).

I hope that you understand my idea and that it will help you :)

user_31147650

Turns out there is no standard way of doing this in Keras yet AFAIK.

See this issue for context.

user_48418320
Related