I am looking to extract and use (within the same Jupyter notebook) the model identified as the best model from RandomizedSearchCV for future fitting and graphing. Specifically, I am looking to re-fit the Keras Neural Network identified as the best so that I can plot the loss and accuracy against the same or other dataset.
If I run the following code, I get the output I expect - the best score and the paramaters used in obtaining that score.
import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline
from skrebate import SURF
from sklearn.model_selection import StratifiedKFold
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
from keras.wrappers.scikit_learn import KerasClassifier
from keras.models import Sequential
from keras.layers import Dense
url = "https://datahub.io/machine-learning/sonar/r/sonar.csv"
dataframe = pd.read_csv(url)
dataset = dataframe.values
X = dataset[:,0:60].astype(float)
y = dataset[:,60]
features, labels = dataset[:,0:60].astype(float), dataset[:,60]
encoder = LabelEncoder()
encoder.fit(y)
encoded_y = encoder.transform(y)
def create_nn_model(input_dims):
# Create model.
model = Sequential()
model.add(Dense(60, input_dim=input_dims, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
# Compile model.
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
param_grid = {'model__epochs': (100,200,300),
'model__batch_size': (10,20,20)}
kfold = StratifiedKFold(n_splits=10, shuffle=True)
for x in range(10, 11, 10): # Iterate through top 10% to 100% of the features.
num_features = int(features.shape[1] * x / 100)
clf = Pipeline([('fs_step', SURF(n_features_to_select=num_features)),
('model', KerasClassifier(build_fn=lambda: create_nn_model(num_features), epochs=100, batch_size=5, verbose=0))])
grid = RandomizedSearchCV(clf, param_grid, n_jobs=-1, cv=kfold, n_iter = 3)
grid_result = grid.fit(features, encoded_y)
print('Best score obtained: {0}'.format(grid_result.best_score_))
print('Parameters:')
for param, value in grid_result.best_params_.items():
print('\t{}: {}'.format(param, value))
I also know how to plot the data I am looking for by building, compiling and fitting a model, as per the following code:
model = Sequential()
model.add(Dense(60, input_dim=60, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(features, encoded_y, epochs=100, batch_size=25, verbose=0)
losses = pd.DataFrame(model.history.history)
import matplotlib.pyplot as plt
losses.plot()
plt.show()
Can I take the best model from the RandomizedSearchCV, fit against data and plot, or must I build, compile and fit based on best_params_? The reason I ask is because I can not identify / access what I understood to be the best model with grid_result.best_estimator_.model. Trying to do so gives:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-31-bbfe0b584f46> in <module>
----> 1 grid_result.best_estimator_.model
AttributeError: 'Pipeline' object has no attribute 'model'
Any help would be appreciated. Thanks.