Basically, I don't understand how the variance was derived here:

$$ \overline{d}_j = \frac{1}{N^2}\sum_{i=1}^N\sum_{i^{'}=1}^N(x_{ij}-x_{i^{'}j})^2 = 2 \cdot var_j $$

That is an equation 14.27 from Elements of Statistical Learning.

Excerpt from section 14.3.3:

The influence of the jth attribute Xj on object dissimilarity D(xi , xi′ ) depends upon its relative contribution to the average object dissimilarity measure over all pairs of observations in the data set $$ \overline{D} = \frac{1}{N^2}\sum_{i=1}^N\sum_{i^{'}=1}^N D(x_{i},x_{i^{'}}) = \sum_{i^{'}=1}^N w_j \cdot \overline{d_j} $$ with $$ \overline{d}_j = \frac{1}{N^2}\sum_{i=1}^N\sum_{i^{'}=1}^N d(x_{ij},x_{i^{'}j}) $$

Using squared-error distance (d(xij, xi'j)) for each coordinate, equation 14.27 can be found.


1 Answer 1


Easing up the notation, let's say we have a set of values ($N$ values) in an array. The following quantity $$\begin{align}\frac{1}{N^2}\sum_i \sum_j (x_i-x_j)^2&=\frac{1}{N^2}\sum_i\left(\sum_j x_i^2-2x_ix_j+x_j^2\right)\\&=\frac{1}{N} \sum_i x_i^2 -\frac{2}{N^2}\sum_i x_i\sum_j x_j+\frac{1}{N}\sum_j x_j^2\\&=\frac{2}{N}\sum_i x_i^2 - 2\bar{x^2}=2\overline {x^2}-2\bar x^2 = \widehat{2\operatorname{var}(x)}\end{align}$$

I have ignored $j$ in your notation as it doesn't contribute to anything, and changed $i'$ to $j$ instead to make notation more clear. I've also ignored the upper and lower limits of the summations, as both are from $1$ to $N$.

The expression is two times the estimate of population variance because $$\operatorname{var}(X)=E[X^2]-E[X]^2$$

  • $\begingroup$ Thanks! That wasn't that hard🙂 $\endgroup$
    – Jedrek369
    Jul 9, 2022 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.