R: NbClust package gives different optimal number of clusters using the same dataset with the random order of rows

Question

I wanna use k-means to figure out the best partition of the Iris dataset, therefore, I used the NbClust function in R to determine the optimal number of clusters. This function uses 30 different indexes to decide the final number of the clusters. I checked that I cleared the working environment and updated R to the latest version (3.6.1) before running the code. However, I found that when I randomized the order of rows, NbClust gave me different suggested numbers of clusters.

My code:

library(NbClust)
data(iris)

data<-iris[,-c(5)]
data.random <- data[sample(nrow(data)),]

set.seed(1) # can be removed
res1<-NbClust(data, distance = "euclidean", min.nc=2, max.nc=10, method = "kmeans", index = "all")  
set.seed(1) # can be removed
res2<-NbClust(data.random, distance = "euclidean", min.nc=2, max.nc=10, method = "kmeans", index = "all")  

res1$Best.nc
res2$Best.nc

The output of the original data suggested that 2 is the optimal number of clusters

* Among all indices:                                                
* 10 proposed 2 as the best number of clusters 
* 8 proposed 3 as the best number of clusters 
* 2 proposed 4 as the best number of clusters 
* 1 proposed 5 as the best number of clusters 
* 1 proposed 7 as the best number of clusters 
* 1 proposed 8 as the best number of clusters 
* 1 proposed 10 as the best number of clusters 

                   ***** Conclusion *****                            
 
* According to the majority rule, the best number of clusters is  2

While the output of the randomized data suggested that 3 is the optimal number of clusters

* Among all indices:                                                
* 8 proposed 2 as the best number of clusters 
* 11 proposed 3 as the best number of clusters 
* 1 proposed 4 as the best number of clusters 
* 1 proposed 5 as the best number of clusters 
* 2 proposed 8 as the best number of clusters 
* 1 proposed 10 as the best number of clusters 

                   ***** Conclusion *****                            
 
* According to the majority rule, the best number of clusters is  3

I also compared the outputs of the indexes between the original and randomized data, parts of indexes changed the suggested number of clusters.

res1$Best.nc == res2$Best.nc

                   KL   CH Hartigan  CCC Scott Marriot TrCovW TraceW
Number_clusters FALSE TRUE     TRUE TRUE  TRUE    TRUE   TRUE   TRUE
Value_Index     FALSE TRUE    FALSE TRUE  TRUE   FALSE   TRUE  FALSE
                Friedman Rubin Cindex   DB Silhouette  Duda PseudoT2
Number_clusters    FALSE FALSE   TRUE TRUE       TRUE FALSE    FALSE
Value_Index        FALSE FALSE   TRUE TRUE       TRUE FALSE    FALSE
                Beale Ratkowsky Ball PtBiserial  Frey McClain  Dunn
Number_clusters FALSE      TRUE TRUE       TRUE FALSE    TRUE  TRUE
Value_Index     FALSE      TRUE TRUE       TRUE FALSE    TRUE FALSE
                Hubert SDindex Dindex  SDbw
Number_clusters   TRUE    TRUE   TRUE  TRUE
Value_Index       TRUE   FALSE   TRUE FALSE

I don't really understand what is causing the output differences for these datasets with randomized order being the only difference between. In my opinion, the optimal number of clusters should be the same, and reproducible (with the slight differences in the value index make sense), especially since the Iris dataset has very distinct 3 groups.

Does anyone have any ideas?

Thanks in advance.

user_13714870 · Answer

NbClust gives you different suggested clusters every time because you are setting the random numbers generator seed only after calling the sample function. I.e. your data.random dataset will always slightly vary, thus impacting the resulting partition.

By just adding set.seed(1) at the very beginning of your code (resetting the random numbers generator's state before calling sample()), the NbClust results become all exactly identical.

To answer the second part of your question, the number of samples you are generating is not statistically significant enough to converge into a consistant partition. Setting a constant seed upfront makes the stochastic process reproducible though.

user_10497812 · Answer

k-means per se will produce different results due to the initialization problem.

Specifically, it picks up (different) random initial points to cluster your data every time you run it.

This might explain why running k-means several times will not show identical results after you have settled the random seed. But, it should be noted that the clusters should be similar.

Otherwise, it might suggest that your data is not meaningful enough to produce several clusters.

R: NbClust package gives different optimal number of clusters using the same dataset with the random order of rows

2 Answers