Using scipy, I noticed that I am allowed to use only Euclidean distance for Ward's method.

Is it because Ward's uses Error Sum of Squared?

What if I use Ward's method with cosine similarity?

Cosine similarity seems still work so, but maybe not perfectly.

What might be the difference?


marked as duplicate by S. Kolassa - Reinstate Monica, kjetil b halvorsen, mdewey, Peter Flom - Reinstate Monica Nov 2 '17 at 12:36

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ Once a metric is published, it becomes kind of reified wrt its inputs and construction. Ward's Method is more than 50 years old which dates its origin back to a time prior to the development of many of the distance metrics in use today. In other words, Euclidean distance was the lingua franca distance metric of the time. This does not mean that you are prohibited from plugging a different distance metric into its algorithm but it does mean that, strictly speaking, you have created a new metric that can no longer be called Ward's Method. $\endgroup$ – Mike Hunter Nov 2 '17 at 11:21
  • $\begingroup$ Then, let's say I use cosine similarity metric to Ward's method. Without any test or evaluation, can you kind of trust that this clustering is meaningful? Do not have to be strict, just want to hear your opinion or intuition. $\endgroup$ – DongukJu Nov 2 '17 at 12:03