0
$\begingroup$

I see both of these equations on Wikipedia and Cross Validated being referred to as "variance":

$\mathrm{Var}(x) = \mathrm{E}\left[ \left( x - \mathrm{E}[x] \right)^2 \right]$

$\frac{1}{n} \sum_i \left(x_i - \bar{x} \right)^2 \quad \text{ where } \quad \bar{x} = \frac{1}{n} \sum x_i$

Does the $\mathrm{E}$ simply imply that the equation in brackets needs to be averaged over all iterations, hence why we place the $\frac{1}{n}$ at the front of the equation?

I just need to be walked through how the top equation allows us to arrive at the bottom equation and both of these are referred to as variance.

Also, is there a term you could use to refer specifically to $\sum_{i=1}^n(x_i-\bar{x})^2$ when not multiplied by $\frac{1}{n}$? Just sum of squares?

$\endgroup$
  • 1
    $\begingroup$ To understand the distinction, you should understand the difference between the population and a sample. $\endgroup$ – Matthew Gunn Oct 6 '18 at 5:15
  • $\begingroup$ I think understand the difference but am still confused. Lets assume the total population was 100 and I measured some variable $x$ in every member of that population. In order to determine the variance of $x$, I would still have to multiply $\sum_{i=1}^n (x_i - \bar{x})^2$ by $\frac{1}{n}$. The top equation does not have a $\frac{1}{n}$. I'm asking is that what the $\mathrm{E}$ implies? $\endgroup$ – Dylan Russell Oct 6 '18 at 5:20
  • $\begingroup$ Try to find the difference between "parameter" and "the estimate of parameter". In your case, the first one is parameter and the second one is the estimate (guess) of the first one. More specifically, the first one is variance of $x$ and the second one is the estimate of the variance of $x$. $\endgroup$ – user158565 Oct 6 '18 at 5:59
4
$\begingroup$

The top equation is theoretical, you cannot use it directly with the data you have. You should have the PDF/CDF to calculate theoretical expected values (i.e. population values). And you should think of population as infinite samples, not 100 as you say.

The second equation is an estimation procedure, so it is an estimation of the variance, i.e. $\hat{var(X)}$. And, in practice, it is usually not equal the theoretical value calculated from the first equation. Moreover, another estimation of variance uses $\frac{1}{n-1}$ instead of $\frac{1}{n}$ because it is unbiased. Please check calculation of $E[S^2]$ in the link.

Finally, yes, the $E$ sign implies taking mean of the inner expression; but they're not equal. For example, approximating $E[g(X)]$ via $\frac{1}{n}\sum{g(x_i)}$ from data is not a rare situation in statistics.

$\endgroup$
  • $\begingroup$ Ahh it all just clicked. Thank you! This is what I needed. $\endgroup$ – Dylan Russell Oct 6 '18 at 6:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.