Find the optimal k(hyperparameter) of KNN given k vs AUC score, such that cv_auc should be maximum and gap between train_auc and cv_auc is minimum

Viewed 31

I have a task to find the optimal hyperparameter(k) of KNN. I plotted the k vs AUC curve using roc_auc_score. I am supposed to find k such that cv_auc is maximum and the gap between train_auc and cv_auc is minimum. How can I achieve that?

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import roc_auc_score
import matplotlib.pyplot as plt

train_auc=[]
cv_auc=[]
k=[i for i in range(1,50,5)]
for i in k:
   knn=KNeighborsClassifier(n_neighbors=i)
   knn.fit(x_train_bow,y_train)  
   y_train_pred=knn.predict_proba(x_train_bow)[:,1]
   y_cv_pred=knn.predict_proba(x_cv_bow)[:,1]

   train_auc.append(roc_auc_score(y_train,y_train_pred))
   cv_auc.append(roc_auc_score(y_cv,y_cv_pred))
              
#plot the roc curve
plt.plot(k,train_auc,label="Train AUC")
plt.plot(k,cv_auc,label="CV AUC")
plt.legend()
plt.xlabel('K:hyperparameter')
plt.ylabel('AUC')
plt.title("Error plot")
plt.show()

picture of the roc curve

print(cv_auc)
print(cv_auc.index(max(cv_auc)))

array1 = np.array(train_auc)
array2 = np.array(cv_auc)
subtracted_array = np.subtract(array1, array2)
subtracted = list(subtracted_array)
print(subtracted)
subtracted.index(min(subtracted))

Output: [0.6241694315220194, 0.6985803616697652, 0.7222662029418654, 0.7429448007376901, 0.7433472984472336, 0.7492335494812746, 0.7499829512940709, 0.7594353468596283, 0.757365782209453, 0.7518153165574067]

7

[0.3758305684779806, 0.1995133667387895, 0.1433755719502956, 0.10953834255228179, 0.09624883964242126, 0.08236753388538032, 0.07710481774180344, 0.06538756093043141, 0.05998659695603492, 0.06576356656762017]

8

0 Answers
Related