Agglomerative Clustering Hierarchy Visualization

user_9090626 0
Viewed 989

How do I visualize the hierarchy formed by the agglomerative clustering as a dendogram. I have a precomputed distance matrix of size (400,400).

clusterer= AgglomerativeClustering(n_clusters=32,affinity="precomputed",linkage="average").fit(distance_matrix)

How do I clearly visualize the result that formed the 32 clusters as a dendogram? I tried visualizing the clusters but since they are 32, the colors were not clearly differentiating them apart.

colors_clusters = clusterer.labels_
fig = plt.figure(figsize=(10,7))
plt.xlabel('median_score', family='Arial', fontsize=9)
plt.ylabel('count_intersections', family='Arial', fontsize=9)
plt.title('Heliopolis', family='Arial', fontsize=12)
plt.scatter(clusters_df['median_score'], clusters_df['count_intersections'], c=colors_clusters, edgecolors='black', s=50)
plt.show()

enter image description here

hierarchical-clustering scikit-learn scipy python
1 Answers

One of the renowned methods of visualization for hierarchical clustering is using dendrogram. You can find a plot example in sklearn library. You can find examples in scipy library as well.

You can find an example from the former link here:

import numpy as np

from matplotlib import pyplot as plt
from scipy.cluster.hierarchy import dendrogram
from sklearn.datasets import load_iris
from sklearn.cluster import AgglomerativeClustering


def plot_dendrogram(model, **kwargs):
    # Create linkage matrix and then plot the dendrogram

    # create the counts of samples under each node
    counts = np.zeros(model.children_.shape[0])
    n_samples = len(model.labels_)
    for i, merge in enumerate(model.children_):
        current_count = 0
        for child_idx in merge:
            if child_idx < n_samples:
                current_count += 1  # leaf node
            else:
                current_count += counts[child_idx - n_samples]
        counts[i] = current_count

    linkage_matrix = np.column_stack([model.children_, model.distances_,
                                      counts]).astype(float)

    # Plot the corresponding dendrogram
    dendrogram(linkage_matrix, **kwargs)


iris = load_iris()
X = iris.data

# setting distance_threshold=0 ensures we compute the full tree.
model = AgglomerativeClustering(distance_threshold=0, n_clusters=None)

model = model.fit(X)
plt.title('Hierarchical Clustering Dendrogram')
# plot the top three levels of the dendrogram
plot_dendrogram(model, truncate_mode='level', p=3)
plt.xlabel("Number of points in node (or index of point if no parenthesis).")
plt.show()
user_37688710
Related