How to plot the K-means and print the points outside the cluster

Viewed 2579

How to plot the K Means for the below data

no,store_id,revenue,profit,state,country
0,101,779183,281257,WD,India
1,101,144829,838451,WD,India
2,101,766465,757565,AL,Japan

My Code is below

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
df1 = pd.get_dummies(df, columns=['state','country'])
clusters = 2   
km = KMeans(n_clusters=8).fit(df1)
labels = km.predict(df1)
df1['cluster_id'] = km.labels_
def distance_to_centroid(row, centroid):
     row = row[['no','store_id','revenue','profit','state','country']]
     return euclidean(row, centroid)
df1['distance_to_center0'] = df1.apply(lambda r: distance_to_centroid(r,
                             km.cluster_centers_[0]),1)

df1['distance_to_center1'] = df1.apply(lambda r: distance_to_centroid(r,
                             km.cluster_centers_[1]),1)



dummies_df =dummies[['distance_to_center0','distance_to_center1','cluster_id']]
test = {0:"Blue", 1:"Red", 2:"Green",3:"Black",4:"Orange",5:"Yellow",6:"Violet",7:"Grey"}
sns.scatterplot(x="distance_to_center0", y="distance_to_center1", data=dummies_df, hue="cluster_id", palette = test)

To find the centre point below is the code

km = KMeans(n_clusters=7).fit(dummies)
closest, _ = pairwise_distances_argmin_min(km.cluster_centers_, dummies)
closest
  • How to scatter plot for the cluster

  • How to get the print points away from the cluster

Like in least outlier method -1 is the outlier(scikit learn).kmeans.labes_ is printing only 1 and 0 , how to get the outliesrs

4 Answers

Since you are clustering based more than 3 features (WD, AL, India, Japan,..) you need to either of the following to visualize the graphs.

  1. Use Dimension Reduction methods like PCA, TSNE or Auto- Encoders into 2 variables (for 2D graph) or 3 variables (3D graph)

  2. Take 2 or 3 features at a time to plot keeping the rest constant.

You can't plot the data for any dimension bigger than 3 (in your case it is 4 without counting the locations). What you could do instead is to find the distances between the features and the K-dots and use that to get a general idea of what is happening. I hope this helps!

Scatter plots for the cluster: You can only plot/visualize in 2D or 3D, but the data in your example has at least 4 features (i.e. 4 dimensions). To plot the clusters you need to reduce the number of dimensions to 2 or 3 as desired, either by reducing the set of features used for the clustering (e.g. by human pre-selection or using feature reduction techniques) and/or projecting the result onto a reduced dimension (e.g. if you have only 3 nodes/clusters, you can use the plane containing all 3 centroid)

Points away from the 'clusters'/Outliers: The Kmeans algorithm will assign every point to a cluster. The compactness of the clusters is reflected in the inertia_ value. One simple method to get outliers is to look at the distance of points from their assigned centroid and use a threshold (like multiple of standard deviation) to classify outliers. Another approach, that is more involved, is to run the algorithm, then remove the data point with the longest euclidean distance to its assigned centroid, rerun the algorithm and check the drop in average intertia. Repeat until there is no significant drop. The set of removed point are the outliers.

There are other clustering algorithms (like DBCSAN) that try to implicitly detect outliers.

Do not listen to nay-sayers. Although it is impossible to reasonably accurately represent the more than 3d data in 2d images, dimensionality reduction techniques like PCA and TSNE can help with that by using a linear (PCA) or non-linear (TSNE) projection on a lower dimensional space.

Here is an example with wine dataset, which has 13 dimensions. The data that you have presented has only 3 datapoints, so it will not be very illustrative.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import scale
from sklearn.cluster import KMeans
from sklearn.datasets import load_wine
from sklearn.decomposition import PCA
import matplotlib.cm as cm

# Load wine dataset. It has 13 float-type features and 3 classes of wine
data = load_wine()

df = pd.DataFrame(data = scale(data.data), columns=data.feature_names)

# Create a separate dataframe for target and predictions
df_labels = pd.DataFrame(data=data.target, columns=['target'])

class_names = list(data.target_names)

# Perform K-means clustering
km = KMeans(n_clusters=3).fit(df)
df_labels['predictions'] = km.predict(df)

# Use PCA to reduce dataset dimentions to project 13 dimensional dataset onto 2 dimensions
pca = PCA(n_components=2).fit(df)

df_reduced = pd.DataFrame(data=PCA(n_components=2).fit_transform(df.values), columns=['x', 'y'])

fig, (ax_true, ax_pred) = plt.subplots(1, 2, figsize=(20, 10))

# Use red, green, and blue to color different clusters
colors = ['r', 'g', 'b']
colors_true = [colors[i] for i in df_labels['target'].values]
colors_pred = [colors[i] for i in df_labels['predictions'].values]

# Plot cluster centroids with X symbols
centroids = pca.transform(km.cluster_centers_)

x = df_reduced['x'].values
y = df_reduced['y'].values
titles = ['target', 'predictions']

for ax, colors, title in zip([ax_true, ax_pred], [colors_true, colors_pred], titles):
    ax.scatter(x, y, marker='o', s=5, linewidths=3, color=colors)

    # Plot the centroids as a black X
    ax.scatter(centroids[:, 0], centroids[:, 1],
                marker='x', s=50, linewidths=2,
                color='black', zorder=10)

    ax.set_title(title, fontsize= 14)
    ax.set_xticks(())
    ax.set_yticks(())

plt.xlim((1.1*min(x), 1.1*max(x)))
plt.ylim((1.1*min(y), 1.1*max(y)))
plt.show()

which should produce a plot like this: enter image description here

On the left the coloring corresponds to the real clusters, on the right it represents KMeans prediction. Comparing the target with prediction in df_labels will reveal the points that were miss-classified (outside any given cluster).

Related