How to speed up Google colab when doing random search in sklearn?

Viewed 485

The following code takes 5.0 minutes to execute on Google colab while on my machine it takes around 3.0 minutes. In all other tasks (machine learning or otherwise) I tested, colab beat my machine by 50-100 %. I tried installing different sklearn versions, running with GPU and also experimenting with n_jobs values but the time either got slower or remained the same.

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.feature_selection import RFE
from sklearn.pipeline import Pipeline
from sklearn.model_selection import RandomizedSearchCV, KFold
from datetime import datetime

param_grid = [
    {'feature_selection': [RFE(estimator=GradientBoostingClassifier(random_state=0))],
     'feature_selection__n_features_to_select': [2],
     'scaling': [StandardScaler()],
     'classification': [GradientBoostingClassifier(random_state=0)],
     'classification__n_estimators': [100, 500],
     'classification__max_features': ['auto', 'log2'],
     'classification__max_depth': [2, 4],
     'classification__learning_rate': [0.01, ],
     'classification__loss': ['exponential'],
     'classification__min_samples_split': [2, 200],
     'classification__min_samples_leaf': [1, 20]},

    {'feature_selection': [RFE(estimator=LogisticRegression(random_state=0))],
     'feature_selection__n_features_to_select': [2],
     'scaling': [StandardScaler()],
     'classification': [LogisticRegression(random_state=0)],
     'classification__C': [0.1, 100, 1000],
     'classification__penalty': ['l1'],
     'classification__solver': ['liblinear']}
]

pipe = Pipeline(steps=[('scaling', StandardScaler()),
                       ('feature_selection', RFE(estimator=GradientBoostingClassifier())),
                       ('classification', GradientBoostingClassifier())])

grid_obj = RandomizedSearchCV(estimator=pipe, param_distributions=param_grid,
                              scoring='neg_brier_score', cv=KFold(shuffle=True), random_state=0,
                              return_train_score=True, n_jobs=-1, verbose=10)

X, y = load_breast_cancer(return_X_y=True)
grid_obj.fit(X, y)

Google colab results:

# [Parallel(n_jobs=-1)]: Using backend LokyBackend with 2 concurrent workers.
# [Parallel(n_jobs=-1)]: Done   1 tasks      | elapsed:   13.3s
# [Parallel(n_jobs=-1)]: Done   4 tasks      | elapsed:   25.8s
# [Parallel(n_jobs=-1)]: Done   9 tasks      | elapsed:  1.0min
# [Parallel(n_jobs=-1)]: Done  14 tasks      | elapsed:  1.4min
# [Parallel(n_jobs=-1)]: Done  21 tasks      | elapsed:  2.2min
# [Parallel(n_jobs=-1)]: Done  28 tasks      | elapsed:  2.8min
# [Parallel(n_jobs=-1)]: Done  37 tasks      | elapsed:  3.8min
# [Parallel(n_jobs=-1)]: Done  46 tasks      | elapsed:  4.6min
# [Parallel(n_jobs=-1)]: Done  50 out of  50 | elapsed:  5.0min finished
0 Answers
Related