I've noticed a rather peculiar but potentially very useful phenomenon when using Scikit-Learn's SVC implementation. Using the built-in rbf kernel with SVC is slower by magnitudes than passing a custom rbf function to SVC().

From what I could see and understand so far, the only difference between the two versions is that in the built-in rbf case, not sklearn but libsvm will compute the kernel. Passing a dedicated kernel function as hyperparameter to SVC() leads to the computation of the kernel inside sklearn, not in libsvm. The results are identical, but the latter case takes only a fraction of the computation time.

Example

I've included an example so that you can replicate this behavior.

I've created a toy dataset that mimics the data I am currently working on. By the way, I also work on data with around a thousand samples but high dimensionality (~50000 features). This results in pretty much the same behavior.

import numpy as np
from time import time
from sklearn.svm import SVC
from sklearn.datasets import make_classification
from sklearn.metrics.pairwise import rbf_kernel
from sklearn.metrics import accuracy_score

# create toy data
n_features = 1000
n_samples = 10000
n_informative = 10
X, y = make_classification(n_samples, n_features, n_informative=n_informative)
gamma = 1 / n_features

Built-in RBF

First, let's fit an SVC using the built-in 'rbf' kernel. This is probably the way people usually run an SVC.

# fit SVC with built-in rbf kernel
svc_built_in = SVC(kernel='rbf', gamma=gamma)
np.random.seed(13)
t1 = time()
svc_built_in.fit(X, y)
acc = accuracy_score(y, svc_built_in.predict(X))
print("Fitting SVC with built-in kernel took {:.1f} seconds".format(time()-t1))
print("Accuracy: {}".format(acc))

Custom RBF function

Second, let's do the same thing only passing the rbf kernel function of sklearn which should do exactly the same.

# fit SVC with custom rbf kernel
svc_custom = SVC(kernel=rbf_kernel, gamma=gamma)
np.random.seed(13)
t1 = time()
svc_custom.fit(X, y)
acc = accuracy_score(y, svc_custom.predict(X))
print("Fitting SVC with a custom kernel took {:.1f} seconds".format(time()-t1))
print("Accuracy: {}".format(acc))

Results

This will give the following result.

Fitting SVC with built-in kernel took 58.6 seconds
Accuracy: 0.9846
Fitting SVC with a custom kernel took 3.2 seconds
Accuracy: 0.9846

My question

1. Does anyone have an idea why passing a kernel function is so much faster than using libsvm's kernel computation?
2. For my specific use case (usually large datasets and long computation time), this actually is super useful as I can run many more hyperparameter settings using the second method since the computation time is so significantly decreased. Any reasons not to do this?