KNN Classifier build from scratch with numpy, what is wrong with the code?

user_12777183 0
Viewed 213

Why my KNN Classifier build from scratch with numpy gives different results than the sklearn.KNeighborsClassifier? What is wrong with my code?

# create a function that computes euclidean distance and return the most common class label 
    # for given k.
    def k_neighbors(self, x):
        lengths = [self.euclidean_length(x, x_train) for x_train in self.X_training] 
        k_index = np.argsort(lengths)[: self.k] 
        k_nearest_labels = [self.y_training[i] for i in k_index] 
        counts = np.bincount(k_nearest_labels) 
        most_common_label = np.argmax(counts) 
        return most_common_label

# running KNN classifier with K=5 to fit the data and make predictions.
classifier1 = KNN_Classifier(k=5) 
classifier1.fit(X_training, y_training)
predicted1 = classifier1.predicting(X_test)

They both apparently do the same but I have different outcomes. Where is the bug in my code?

from sklearn.neighbors import KNeighborsClassifier

classifier2 = KNeighborsClassifier(n_neighbors=5, algorithm='brute', p=2)
classifier2.fit(X_training, y_training)
predicted2 = classifier2.predict(X_test)
knn scikit-learn numpy python
1 Answers

Based on sklearn documentation, there are multiple reasons:

  1. Distance metric: you are using Euclidean distance metric, while sklearn by default uses minkowski which in X,Y make differences
  2. To find k nearest neighbours, sklearn, by default, choose one of the kd_tree, BallTree and BruteForce methods, however, in your k_neighbours() function, you use BruteForce.
  3. Last but not least, k value in your test is 5, while you're using 4 for skleran equivalent
user_32796030
Related