Tuning hyperparameters with Keras return different results depending on the method used

Viewed 219

I am tuning my model hyperparameters using the Keras tuner tool as follows:

tuner.search(
    x=preprocessed_train_df,
    y=preprocessed_train_df,
    validation_split=0.1,
    epochs=10,
    callbacks=[es],
)

I tested the tuner results using the 2 following methods, which theoretically should return into the same result:

Method 1:

best_hps = tuner.get_best_hyperparameters(num_trials=1)[0]
best_model1 = tuner.hypermodel.build(best_hps)

Method 2:

best_model2 = tuner.get_best_models(num_models=1)[0]

When I run best_model1.summary() or best_model2.summary() I get the same summary results but If I fit the model or evaluate it as follows:

loss, loss2, accuracy = best_model1.evaluate(preprocessed_train_df, preprocessed_train_df)
loss_, loss2_, accuracy_ = best_model2.evaluate(preprocessed_train_df, preprocessed_train_df)

I get different loss results, which suggests that best_model1 and best_model2 are somehow different.

1 Answers

It seems that the method

BaseTuner.get_best_models(num_models=1)

returns a model that has been already trained using the number of epochs defined in tuner.search.

While the method

best_hp = tuner.get_best_hyperparameters()[0]
model = tuner.hypermodel.build(best_hp)

starts to train the model from 0. This second approach is the one recommended by Keras. Hier the source.

Related