Number of terms within a k-means cluster

user_13728565 0
Viewed 102

I need to know the number of terms within a cluster generated by k-means.

I can even find a possible resolution, content is failing, on this link.K-Means Clustering - output clusters contains same number of elements but in different order [ Python ]

Below is my code:

vect = TfidfVectorizer(input='filename', stop_words=stopwords_list, encoding='utf8', ngram_range=(1,2), min_df=3)
tf_idf_matrix = vect.fit_transform(list_of_files)
print(tf_idf_matrix)
kmeans_6 = KMeans(n_clusters=17, n_init=10, init="k-means++", max_iter=100)
clustter6 = kmeans_6.fit_predict(tf_idf_matrix)
clustter6
order_centroids = kmeans_6.cluster_centers_.argsort()[:, ::-1]
termos = list(vect.vocabulary_)
clusterlists = []
for i in range(17):
dummy_list  = []
  for ind in order_centroids[i]:
    dummy_list.append('%s' % termos[ind])
  clusterlists.append(dummy_list)

The output of the above code looks like this,

clusterlists[0] = 52005 examples
clusterlists[1] = 52005 examples.

I would like, for example, clusterlists [0] to present, for example, 3000 examples. Clusterlists 1 presents, for example, 3150 examples. However, it is showing the total value of terms in the database for all indexes of the clusters.

scikit-learn k-means cluster-analysis python-3.x python
2 Answers

KMeans has a labels_ attribute that, for each sample, stores the cluster number the sample was assigned to. You can use np.unique() to count up those numbers:

import numpy as np

print(np.unique(kmeans_6.labels_, return_counts=True))

This will print two arrays: first the list of cluster id numbers, and then the corresponding counts.

user_130141720

To find the number of samples/examples that are present in each centroid (cluster), you first create an array of zeros that represents the number of k centers as follows:

clusterlists = np.zeros(len(order_centroids))  # len = 17 in your case

Where each index of the array corresponds to a unique centroid/cluster. (eg: index 0 = centroid #0, index 1 = centroid #1 ...)

Now to have each entry in the array represent the number of members in each cluster you can add +1 to the index corresponding to the cluster that the encountered sample belongs into:

for i in clustter6:
    clusterlists[i] += 1
print(clusterlists)
user_144920010
Related