I'm using skopt BayesSearchCV, although I previously had the same issue using Keras-Tuner Bayesian Oracle. I'm trying to optimize the hyperparameters of an SVC. There are 1200 data points in the train dataset with only 5 features each, so dataset size shouldn't be the issue. Some of the models train in a fraction of a second while some just never finish training, so I assume the bounds for my hyperparameters need to be adjusted. Currently I have:
model = pipeline.Pipeline([('scaler', StandardScaler()), ('model', SVC(cache_size=500))])
search_params = {
"model__C": Real(1e-4, 1e4, prior='log-uniform'),
"model__kernel": Categorical(['linear', 'poly', 'rbf', 'sigmoid']),
'model__degree': Integer(2, 8),
'model__gamma': Real(0.01, 0.9),
'model__coef0': Real(0.001, 1000, prior='log-uniform'),
'model__class_weight': Categorical([None, 'balanced'])
}
searchcv = BayesSearchCV(
estimator=model,
search_spaces=search_params,
n_iter=100,
n_jobs=-1,
cv=5,
refit=True,
error_score=0,
scoring=metrics.make_scorer(metrics.balanced_accuracy_score),
return_train_score=True,
verbose=1
)
searchcv.fit(x_train, y_train)
How should I change my hyperparameter spaces so that it doesn't get stuck training a single model for hours