How to evaluate K-Means Clustering since automatic indexes of clusters don't match true labels?

user_12065308 0
Viewed 1059

How do we measure the accuracy of a K-Means clustering algorithm (say, generate a confusion matrix) since the automatic indexes of cluster is probably a permutation of the original labels?

multilabel-classification scikit-learn k-means cluster-analysis python
2 Answers

I don't exactly know what you mean too. Your original labels perhaps is the ground truth labeling. The clustering results provided by k-means is usually an integer with range given as many as the k clusters you wish the k-means algorithm to give you.

I typically use pandas.crosstab function to visualize the localizations of the groundtruth labeling with kmeans labeling with cross-tabulation.

For better visualization, you may want to use the following:

import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(30,10))

# plot the heatmap for correlation matrix
ax = sns.heatmap(crosstab_groundtruth_kmeans.T, 
                square=True, annot=True, fmt='.2f')

ax.set_yticklabels(
    ax.get_yticklabels(),
    rotation=0);

out: enter image description here

Good luck!~

user_67255380

k-means is a clustering (grouping algorithm, not used for classification), hence, it is not feasible to check and analyze accuracy. Major concept of k-means is to find a cluster of data-points which maximize the "between-cluster" distance (and does not have the concept of labels, and hence, you can't get accuracy matrix). More insights: https://scikit-learn.org/stable/modules/clustering.html#k-means

The accuracy (assuming, you want to visualize which cluster consists of which data points) has to be analyzed manually using the predict method from sklearn.cluster.KMeans. It basically "Predicts the closest cluster each sample in X belongs to." (from documentation)

user_91192020
Related