Intuitive explanation of Ward's method

Question

I got this explanation of the Ward's method of hierarchical clustering from Malhotra et. al (2017), and I don't really get what it means:

Ward’s procedure is a variance method which attempts to generate clusters to minimise the within-cluster variance. For each cluster, the means for all the variables are computed. Next, for each object, the squared Euclidean distance to the cluster means is calculated. These distances are summed for all the objects. At each stage, the two clusters with the smallest increase in the overall sum of squares within cluster distances are combined.

I understand that if we have $n$ objects, then we start off with $n$ clusters. From there, they say that the means for all the variables is computed (so we got $p$ x $n$ means). And then they say that the distance from each object to to the 'cluster means' is calculated. This is where I get confused - the object means are the cluster means (at stage 1), so what distance is there to calculate?

Thanks a mil!

Answer 1

You are right, at the first stage, where each point $x\in X$ in your dataset $X$ is a cluster, all the distances to the cluster means are zero, so the overall sum of squared within-cluster distances is zero. So now the task is to find the two points which to combine first, such that afterward the overall sum of squared within-cluster distances is the least increased, i.e. you have to find, at stage two, the two points: $$ (x_k, x_l) = argmin_{(x_r, x_s)\in X^2} \{\|x_r - x_s\|\}. $$ You iterate as described, but in those later stages, the cluster centers are not necessarily anymore the points themselves.