Custom SVM Polynomial kernel sklearn

Question

I got asked as an assignment to develop a custom polynomial (degree = 3,4,5) kernel for SVM and compare its accuracy to the in-built poly kernel of the sklearnkit (should be almost the same) I tried to follow the polynomial kernel definition but my results seems not to be quite similar, this is my code:

def poly_kernel_fn(X, Y):
# Implement a polynomial kernel
#  - args: 2 numpy arrays of shape [n_samples, n_features]
#  - returns: computed kernel matrix of shape [n_samples, n_samples]

K = np.zeros((X.shape[0],Y.shape[0]))
K = (X.dot(Y.T) + 1)**4
return K

clfpoly = svm.SVC(kernel='poly', degree=4)
clfpoly.fit(X_train, y_train)
zpoly = clfpoly.predict(X_test)
print("The accuracy with in-built 3D polynomial kernel is: ",accuracy_score(y_test, zpoly)*100,"%")

clf = svm.SVC(kernel=poly_kernel_fn)
clf.fit(X_train, y_train)
z = clf.predict(X_test)
print("The accuracy with custom rbf kernel is: ",accuracy_score(y_test, z)*100,"%")

The accuracy results is the following:

The accuracy with in-built 4D polynomial kernel is: 56.99999999999999 %
The accuracy with kernel is: 59.0 %

If I change the polynomial grade to 3 or 5 it changes even more, so I do not know if I am doing something wrong or simply is not possible to match the in-built accuracy.

Thanks for your help

user_5612259 · Answer

You have to look at the definition of the poly case at

https://scikit-learn.org/stable/modules/svm.html#svm-kernels

coef = 0 and gamma = 1/(n_features*.var()) Then you can get the same

from sklearn.datasets import make_classification
from sklearn import svm
import numpy as np

gamma = None
   
def poly_kernel_fn(X, Y):
   
   K = np.zeros((X.shape[0],Y.shape[0]))
   K = (gamma*X.dot(Y.T))**4
   return K
   
   

if __name__=="__main__":
    
    X, Y = make_classification(10, 5)  # random data
    
    clf1 = svm.SVC(kernel='poly', degree=4) # built in function
    
    clf1.fit(X, Y)
    print("built in score  = ", clf1.score(X,Y))
    
    gamma = 1/(5*X.var())
    
    clf2 = svm.SVC(kernel=poly_kernel_fn)
    
    clf2.fit(X, Y)
    print("custom in score = ", clf2.score(X,Y))

In [9]: run main.py
built in score  =  0.8
custom in score =  0.8

Custom SVM Polynomial kernel sklearn

1 Answers