i'm approaching a clustering problem using k-means on my laptop (using jupyter-lab).
As first step, after data pre-processing, i'm trying to understand the optimal number of cluster to choose, by calculating silhouette and elbow measures. So i'm running this code:
test = df_std
wcss_cust = []
sil_score = []
n_clusters = range(2, 9)
for i in n_clusters:
clusterer = KMeans(n_clusters = i, init = 'k-means++', random_state = 42 , max_iter=150)
cluster_labels = clusterer.fit_predict(test)
silhouette_avg = silhouette_score(test, cluster_labels)
wcss_cust.append(clusterer.inertia_/1000)
sil_score.append(silhouette_avg*1000)
print("For n_clusters =", i, "The average silhouette_score and wcss are : silhouette_score= ", silhouette_avg.round(decimals=2) , " , wcss: " , clusterer.inertia_ )
Note: Please condiser that my dataframe has 250k rows each of one is a customer i want to be clustered. The number of feature that i'm using is not that high (just 4 numeric measures)
The problem i'm having is that: the code is running wiredly slow. my python is 64 bit like my laptop
