1
$\begingroup$

I've been out of academia for the past few years, so my knowledge of statistics is very rusty and I would love some help with my interpretation.

I'm trying to validate my hierarchical clustering solution produced by using wards/squared Euclidean distance (SPSS). I'm trying to replicate this in R to validate my solution using NBClust, and I think I need to use Ward.D2 to take into account I used Euclidean distance square in the original clustering analysis.

I have been researching all I can to confirm this but I find the language really difficult to understand and worry I am missing something.

This from the forum: What algorithm does ward.D in hclust() implement if it is not Ward's criterion?

"It basically boils down to the fact that the Ward algorithm is directly correctly implemented in just Ward2 (ward.D2), but Ward1 (ward.D) can also be used, if the Euclidean distances (from dist()) are squared before inputing them to the hclust() using the ward.D as the method. For example, SPSS also implements Ward1, but warn the users that distances should be squared to obtain the Ward criterion. In such sense implementation of ward.D is not deprecated, and nonetheless it might be a good idea to retain it for backward compatibility."

And from From Murtagh & Legendre 2014

"This article has shown that when they are applied to the same dissimilarity matrix, only Ward2 minimizes the Ward clustering criterion and produces the Ward method. The Ward1 and Ward2 algorithms can be made to optimize the same criterion and produce the same clustering topology by using Ward1 with squared distances, and Ward2 with the distances themselves. Furthermore, taking the square root of the clustering levels produced by Ward1 used with squared distances produces the same clustering levels as Ward2 used with the distances themselves."

My question: So am I correct in my reading of these references that my original clustering using Ward1 with the selection of Euclidean distance squared is THE SAME AS Ward2 using Euclidean distances?

Thanks in advance.

$\endgroup$
  • $\begingroup$ Yes. If you are speaking about the results of the clustering. However, the look of the dendrogram depends on which distance, squared or not, and cumulated or not, is plotted on Y axis. Therefore, depending on the software and input parameters, dendrogram from Ward1 and Ward2 might look different while monotonically equivalent due to same results. Note that Ward1 (which demands squared eucl. d) is the original algorithm and is faster. $\endgroup$ – ttnphns Nov 22 '17 at 8:39
  • $\begingroup$ Please read also in here: stats.stackexchange.com/a/63549/3277; stats.stackexchange.com/a/217742/3277. $\endgroup$ – ttnphns Nov 22 '17 at 8:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.