I am applying ward hierarchical clustering on a data set for which I have pairwise similarities. Since hierarchical clustering need a dissimilarity matrix, I am trying to convert my similarity matrix into a dissimilarity one. Besides, ward algorithm needs a euclidian dissimilarity, so I tried several conversions like the one proposed here Warning: ward's linkage specified with non-Euclidean dissimilarity matrix

And, when I compute the cophenet, I get a poor value (around 0.39, smaller than all the values I previously got).

How can I convert my spearman similarity matrix into a euclidian dissimilarity one?

  • 1
    $\begingroup$ It is geometrically correct to convert angular similarity (such as Pearson correlation) to euclidean distance by the law of cosines, as your link shows. But Spearman rho is Pearson r for ranks, so euclidean distance will also express ranked data, not original data. BTW, rho's traditional computational formula is explicitly a conversion of the distance back into the similarity. $\endgroup$ – ttnphns Apr 3 '14 at 13:20
  • $\begingroup$ Saying 0.39 cophenetic r is "poor" is just because fear has big eyes. Your clustering structure isnt' too strong this time. So what? Keep on living. May try other clustering method or other clustering criterion. $\endgroup$ – ttnphns Apr 3 '14 at 13:26
  • $\begingroup$ Is this conversion (distance back into similarity) valid only when there are no ties between X and Y? (the distance D appears in the second formula given in the link) $\endgroup$ – bigTree Apr 3 '14 at 13:30
  • $\begingroup$ More general formula of rho is here, page 641. But computing rho as Pearson r, after ranking, is the also valid equivalent formula. It conversion to euclidean d by "law of cosines" is also ever valid. $\endgroup$ – ttnphns Apr 3 '14 at 13:50
  • $\begingroup$ @ttnphns: do you want to post your comment(s) as an answer? Better to have a short answer than no answer at all. Anyone who has a better answer can post it. $\endgroup$ – Sycorax Aug 12 '18 at 3:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.