10
$\begingroup$

Consider the context of a dendrogram clustering. Let us call original dissimilarities the distances between the individuals. After constructing the dendrogram we define the cophenetic dissimilarity between two individuals as the distance between the clusters to which these individuals belong.

Some people consider that the correlation between the original dissimilarities and the cophenetic dissimilarities (called cophenetic correlation) is a "suitability index" of the classification. This sounds totally puzzling to me. My objection does not rely on the particular choice of the Pearson correlation, but on the general idea that any link between the original dissimilarities and the cophenetic dissimilarities could be related to the suitability of the classification.

Do you agree with me, or could you present some argument supporting the use of the cophenetic correlation as a suitability index for the dendrogram classification ?

Improve this question
$\endgroup$
13
  • $\begingroup$ You don't explain your objection to the (quite intuitive)general idea that any link between the original dissimilarities and the cophenetic dissimilarities could be related to the suitability of the classification. Classification should reflect original dissimilarities. Dendrogramic classification's basic feature to do this is via cophenetic dissimilarity. Is there smth. wrong? $\endgroup$
    – ttnphns
    Commented Jul 26, 2012 at 11:24
  • 1
    $\begingroup$ By the way, one should not mix concept of hierarchical (agglometative) clustering with hierarchical (dendrogramic) classification. The clustering produces its dendrogram as a process report; it doesn't claim it to be hierarchical classification result. $\endgroup$
    – ttnphns
    Commented Jul 26, 2012 at 11:32
  • 1
    $\begingroup$ Cophenetic correlation was proposed for "dogmatic" classifications only - where the classification should reflect pairwise dissimilarities, thence the notion of usefulness of (cophenetic) correlation follows immideately. $\endgroup$
    – ttnphns
    Commented Jul 26, 2012 at 11:51
  • 2
    $\begingroup$ You might want to read this paper on cophenetic correlation $\endgroup$
    – ttnphns
    Commented Jul 26, 2012 at 11:54
  • 3
    $\begingroup$ @StéphaneLaurent I have nothing to contribute as an answer to your question but I have been reading the dialog. Nothing you said sounded offensive to me. Also you said you didn't know the difference between classification and clustering and I haven't seen that simple question answered. It is the differece between what the machine learning people call supervised and unsupervised learning. In classification you know all the class labels for your data and use that information to construct a classification rule for future cases that don't have labels. In cluster you have no labelling. $\endgroup$
    – Michael R. Chernick
    Commented Jul 26, 2012 at 18:45

1 Answer 1

Reset to default
2
$\begingroup$

... is a "suitability index" of the classification

To me it's not right clear what is meant by that. The way I got it, is that

the correlation between the original dissimilarities and the cophenetic dissimilarities (called cophenetic correlation)

is a measure of the hierarchical structure among the observations, i. e. their distances. That is to say the dissimilarities to observations in a different cluster are preferably similar. Considering to datasets A and B clustered using euclidean distance and complete linkage... enter image description here ...even without having a look at the cophenetic distance map or computing cophenetic correlation, one can see, that the cophenetic correlation of A is higher than that of B. In a hierarchy there are levels. So the CC tells about whether distances to observations on the same level (cluster) are similar.

For the sake of completeness: The cophenetic correlations are CC(A) = 0.936 and CC(B) = 0.691

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ I wish I were more expert on this. I don't quite follow your example w/ the heatmaps. What is it that you see that makes it obvious the CC(A) > the CC(B)? Eg, if the upper triangles were cophenetic distances & the lower triangles were original distances, & both displayed similar patterns, then I would recognize that the CC would be high, etc. W/ these I'm not sure how to make such an inference. Is it just that A will naturally give rise to better clustering & so the resulting CC will just have to end up matching well? $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 25, 2014 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.