0
$\begingroup$

I've gone ahead and clustered a dataset using a Euclidian Hierarchical Clustering algorithm:

from scipy import cluster

distance_metric = 'euclidean'
linkage_matrix = cluster.hierarchy.linkage(X_norm_not_missing, method='single', metric=distance_metric)

I'm then calculating the Cophenetic Coefficient in order to determine the goodness of fit of the clustering:

from scipy.spatial.distance import pdist

cophenetic_corr_coef, _ = cluster.hierarchy.cophenet(linkage_matrix, pdist(X_norm_not_missing))
cophenetic_corr_coef

However, this calculates the values using the full hierarchical cluster, rather than a pruned one. When I go ahead and plot it, for example, I can specify a p value to truncate the dendogram:

cluster.hierarchy.dendrogram(linkage_matrix, 
                             leaf_rotation=90,
                             leaf_font_size=12,
                             # no more than p levels of the dendogram tree are displayed
                             truncate_mode='level', 
                             p=12,
                            )

However, I'm not seeing a way to prune the actual model/linkage matrix, rather than simply the depiction of the dendogram of the linkage matrix. How can I go ahead and prune the hierarchical cluster that has been generated in order to calculate the Cophenetic Coefficient for a truncated Hierarchical Clustering algorithm?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.