Display cluster labels for a scipy dendrogram

Question

I'm using hierarchical clustering to cluster word vectors, and I want the user to be able to display a dendrogram showing the clusters. However, since there can be thousands of words, I want this dendrogram to be truncated to some reasonable valuable, with the label for each leaf being a string of the most significant words in that cluster.

My problem is that, according to the docs, "The labels[i] value is the text to put under the ith leaf node only if it corresponds to an original observation and not a non-singleton cluster." I take this to mean I can't label clusters, only singular points?

To illustrate, here is a short python script which generates a simple labeled dendrogram:

import numpy as np
from scipy.cluster.hierarchy import dendrogram, linkage
from matplotlib import pyplot as plt

randomMatrix = np.random.uniform(-10,10,size=(20,3))
linked = linkage(randomMatrix, 'ward')

labelList = ["foo" for i in range(0, 20)]

plt.figure(figsize=(15, 12))
dendrogram(
            linked,
            orientation='right',
            labels=labelList,
            distance_sort='descending',
            show_leaf_counts=False
          )
plt.show()

Now let's say I want to truncate to just 5 leaves, and for each leaf, label it like "foo, foo, foo...", ie the words that make up that cluster. (Note: generating these labels is not the issue here.) I truncate it, and supply a label list to match:

labelList = ["foo, foo, foo..." for i in range(0, 5)]
dendrogram(
            linked,
            orientation='right',
            p=5,
            truncate_mode='lastp',
            labels=labelList,
            distance_sort='descending',
            show_leaf_counts=False
          )

and here's the problem, no labels:

I'm thinking there might be a use here for the parameter 'leaf_label_func' but I'm not sure how to use it.

user_15840081 · Answer

you can simply write:

hierarchy.dendrogram(Z, labels=label_list)

Here is a good example, using pandas Data Frame :

import numpy as np
import pandas as pd
from scipy.cluster import hierarchy
import matplotlib.pyplot as plt

data = [[24, 16], [13, 4], [24, 11], [34, 18], [41, 
6], [35, 13]]
frame = pd.DataFrame(np.array(data), columns=["Rape", 
"Murder"], index=["Atlanta", "Boston", "Chicago", 
"Dallas", "Denver", "Detroit"])

Z = hierarchy.linkage(frame, 'single')
plt.figure()
dn = hierarchy.dendrogram(Z, labels=frame.index)

user_15111286 · Answer

Seems to me that @coradek answer have a little mistake, though it was very helpful

I used his code (with df as pandas DataFrame) with correction:

plt.figure(figsize=(20,10))
labelList = df.apply(lambda x: f"{x['...']}",axis=1)
Z = linkage(df[["..."]])
R = dendrogram(Z,no_plot=True)
labelDict = {leaf: labelList[leaf] for leaf in R["leaves"]}
dendrogram(Z,leaf_label_func=lambda x:labelDict[x])
plt.show()

because the code presented above always gave me the same order of ticks

Display cluster labels for a scipy dendrogram

3 Answers