What is the logic behind Weaviate's KNN classification algorithm?

Viewed 119

I am using Weaviate's KNN engine to perform multiclass classification, but I do not understand:

  • What kind of KNN is it using?
  • is it using simple voting or distance weighted voting?
  • is it using cosine distance?
  • does it use a method to find possible neighbors or use brute force to find all true nearest neighbors?
  • what are winning/losing groups and how are they used to predict the class of a new vector?
1 Answers

Great questions, let me answer them one-by-one:

What kind of KNN is it using?

I'm not entirely sure what you mean by "what kind", but I think this will be answered through the remaining questions. One thing to keep in mind is that Weaviate's kNN classification makes use of the existing vector index for a particular class. So depending on whether you brought your own vectors or used a model for vectorization, the input parameters will determine what the classification runs on. For example, if you are using a text2vec module on a text property with the name description, then those descriptions will be used to find the closest matches.

is it using simple voting or distance weighted voting?

As of v1.8.0 it's simple voting. Distance weighted voting might be a nice addition for the future.

is it using cosine distance?

As mentioned in the first answer, whatever settings you chose for your class will also apply to the classification. As of v1.8.0 Weaviate typically uses cosine distance, but there are plans to add other distance metrics in the near future.

does it use a method to find possible neighbors or use brute force to find all true nearest neighbors?

Similarly to above, it follows the settings of the class. Typically it uses an HNSW approximate index. You can tune its parameters on a class level.

what are winning/losing groups and how are they used to predict the class of a new vector?

They are a tool to gain some insights into why a classification resulted in the way that it did. The winning group is the "group" of results that had the highest votes and therefore lead to the item being classified as such. The losing groups (there can be multiple) are alternative "groups" which didn't get the highest vote. Knowing each group might be a good insight to tweak k for future classifications.

Related