4
$\begingroup$

Ward (1963) provides a commonly used criterion for hierarchical clustering. It's based on the following definition (p. 237):

Given a set of ratings for 10 individuals, $\{2, 6, 5, 6, 2, 2, 2, 0, 0, 0\}$, a common practice is to use the mean value to represent all the scores rather than to consider individual scores. The "loss" in information that results from treating the 10 scores as one group with a mean of 2.5 can be indicated by a "value-reflecting" number, the error sum of squares (ESS).

The error sum of squares is given by the functional relation,

$$\text{ESS} = \sum_{i=1}^n x_i^2 - \frac{1}{n}\left( \sum_{i=1}^n x_i \right)^2$$

where $x_i$ is the score of the $i$th individual. The ESS for the example is […] 50.5.

If somebody asked me how to quantify the loss of information incurred by representing a vector with its mean, I'd say the SD or variance. Or if you wanted the sum of squares rather than the mean of squares, you'd multiply the variance by the sample size, and get $\sum_{i=1}^n (x_i - \bar{x})^2$. This is the sum of squared distances from the mean. So why would one use Ward's ESS instead of one of these quantities?

Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58(301), 236–244. doi:10.2307/2282967. Retrieved from https://web.archive.org/web/20050312103440/http://iv.slis.indiana.edu/sw/data/ward.pdf

$\endgroup$
  • $\begingroup$ But it is just the same as your formula of ESS. Many statistical programs compute ESS and then variance (or covariance) the way shown by Ward, i.e. without centering of the data values. It is faster. See footnote 1 here. $\endgroup$ – ttnphns Nov 14 '17 at 22:07
  • $\begingroup$ See also the last sentence in here stats.stackexchange.com/a/237811/3277. Within-group scatter (or scatter matrix), that is, SSerror, or W, can be computed without centering the data. $\endgroup$ – ttnphns Nov 14 '17 at 22:19
4
+100
$\begingroup$

\begin{align} \operatorname{Var}(\vec x) \propto \sum_{i=1}^n(x_i - \bar x)^2 &= \sum_i x_i^2 - 2\bar x \sum_ix_i + n \bar x^2 \\ &= \sum_i x_i^2 - n \bar x^2 = \sum_i x_i^2 - \frac 1n \left(\sum_i x_i\right)^2 = \text{ESS}. \end{align}

I think $ESS$ is more sensible when talking about compression because $ESS = ||\vec x - \bar x \mathbf 1||^2_2$ so this coincides with the usual norm on $\mathbb R^n$.

$\endgroup$
  • $\begingroup$ +1. This answer would look even more classy without "Does that answer your question or did I miss something?" :-) $\endgroup$ – amoeba Nov 14 '17 at 21:55
  • $\begingroup$ @amoeba i didn't mean that to be rude, i was just concerned that i missed some aspect of the question. Also thank you for the better formatting. $\endgroup$ – jld Nov 14 '17 at 21:57
  • 1
    $\begingroup$ I did not mean to say that it was rude! Merely that an answer without any words looks cool IMHO. But of course additional explanations are welcome nevertheless. $\endgroup$ – amoeba Nov 14 '17 at 22:00
  • 1
    $\begingroup$ @amoeba ah i see what you mean :) sort of like the infamous W: math.stackexchange.com/questions/74347/… $\endgroup$ – jld Nov 14 '17 at 22:02
  • $\begingroup$ Hahaha did not see that one. $\endgroup$ – amoeba Nov 14 '17 at 22:04
2
$\begingroup$

Ward's ESS is the same as the SS you mention. If you distribute the terms in your formula you get:

$ \sum(x_i - \bar x)^2 = \sum x_i^2 + \sum \bar x^2 - 2 \bar x \sum x_i = \sum x_i^2 - n \bar x ^2 = \sum x_i^2 - (\sum x_i)^2 / n$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.