0
$\begingroup$

If we have a set of samples $x_1 ,\dots, x_N$ and we denote with $\bar x = \frac{1}{N} \sum_i x_i$ their average, then the sample variance is defined as

$s^2=\frac{1}{N-1} \sum(x_i - \bar x)^2$

(see [1], for example).

I have found that someone says that the sample estimate of the variance is $s^2=\frac{1}{N^2}\sum_i \sum_{i'} (x_i - x_{i'})^2 $

(see formula (14.27) of [2])

"Sample variance" and "sample estimate of the variance" should be the same thing, right? However, I don't find how the 1st formula equates the 2nd.

Has anyone any idea? Thanks!

[1] https://onlinecourses.science.psu.edu/stat414/node/66

[2] Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). http://statweb.stanford.edu/~tibs/ElemStatLearn/

$\endgroup$
5
  • $\begingroup$ The second equation appears to be variance across stratas $\endgroup$
    – Jon
    Commented May 17, 2017 at 16:23
  • $\begingroup$ The second estimate involves a double summation and it is not clear exactly how $x_i$ differs from $x_i$,. What ranges are the summations taken over? Finally the choice of N versus N-1 in the denominator in the case of equation (1) just depends on whether or not you want to use the unbiased estimator. $\endgroup$ Commented May 17, 2017 at 16:29
  • $\begingroup$ @MichaelChernick the double summation is over the same vector twice. $\endgroup$
    – AdamO
    Commented May 17, 2017 at 16:54
  • $\begingroup$ Maybe you missed "2" when you copied formula (14.27) of [2]. (2*var_j) $\endgroup$
    – user158565
    Commented May 17, 2017 at 17:59
  • $\begingroup$ See stats.stackexchange.com/questions/225734/… $\endgroup$
    – Glen_b
    Commented May 18, 2017 at 0:49

1 Answer 1

2
$\begingroup$

The expression is missing a factor of 2.

Using the ol' add and subtract method to incorporate the term that you want:

\begin{eqnarray} \sum_{i=1}^n \sum_{i^\prime}^n \left( x_i - x_{i^\prime} \right)^2 &=& \sum_{i=1}^n \sum_{i^\prime}^n \left( x_i - \bar{x} + \bar{x} - x_{i^\prime} \right)^2 \end{eqnarray}

Then we know we can factor the quadratic term because we like it that way.

\begin{eqnarray} \sum_{i=1}^n \sum_{i^\prime}^n \left( x_i - \bar{x} + \bar{x} - x_{i^\prime} \right)^2 &=& \sum_{i=1}^n \sum_{i^\prime}^n \left( \left( x_i - \bar{x} \right)^2 + \left(x_{i^\prime} - \bar{x} \right)^2 + 2 \left(x_{i^\prime} - \bar{x} \right)\left(x_{i} - \bar{x} \right) \right) \end{eqnarray}

The cross-product goes to 0.

\begin{equation} \sum_{i=1}^n \sum_{i^\prime}^n \left( \left( x_i - \bar{x} \right)^2 + \left(x_{i^\prime} - \bar{x} \right)^2 \right) = \sum_{i=1}^n \left( x_i - \bar{x} \right)^2 + \sum_{i^\prime}^n \left(x_{i^\prime} - \bar{x} \right)^2 = 2 \sum_{i}^n \left( x_i - \bar{x} \right)^2 \end{equation}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.