When I calculate the variance by hand I get something different than Rstudio.

They guy in the video however, did calculate it as I did, wrong. Why is that?

My calculations:

Observations $=1,2,3,4,5,6,7,8,9$

Calculation by hand: $$E[X] = \frac{1}{n}*\sum^n_1 x_i = \frac{45}{9} = 5 \\\text{or}\\ E[X] = \sum x_i *\frac{1}{9} = 5 \\ Var(X) = E[(X-\mu)^2]=\sum (x_i - \mu)^2* \frac{1}{9} = \frac{20}{3} \\\text{or}\\ Var(X) = \left(\frac{1}{n} * \sum x_i^2\right)-\mu^2 = \frac{20}{3}$$

However when I use the following code in R i get different results.

a <- c(1:9)
mean(a) % = 5
var(a) % = 7.5


What is happening here\Why are the results different?
Are the formulas I used for the calculation by hand correct?

  • 3
    $\begingroup$ The var function in R estimates the sample variance, which is calculated as $$Var(X)=\dfrac{1}{n-1}\sum_i (x_i-\bar{x})^2$$. $\endgroup$ – user2974951 Feb 18 '19 at 14:16
  • $\begingroup$ Ah I see. Which method is correct? $\endgroup$ – Jürgen Erhardt Feb 18 '19 at 14:18
  • $\begingroup$ The formula you presented looks like is estimating the population variance, which is similar except divided by $n$ instead of $n-1$, also using the population mean $\mu$, rather than the sample mean $\bar{x}$. Which one you choose depends on whether you have a sample or population data. As $n$ gets larger this becomes less important (in general). $\endgroup$ – user2974951 Feb 18 '19 at 14:20
  • $\begingroup$ I see. So $\frac{1}{n}$ for sample data and $\frac{1}{n-1}$ for the whole population data? $\endgroup$ – Jürgen Erhardt Feb 18 '19 at 14:24
  • 3
    $\begingroup$ @user2974951, "sample variance" is a jargon. It isn't "variance in the sample", it is estimating population variance. The "n-1" denominator makes this estimator unbiased (given that the mean with use is a sample mean). The "n" denominator makes the estimator biased, it is called maximum likelihood estimate of population variance. $\endgroup$ – ttnphns Feb 18 '19 at 14:43

The formula suggests the author is estimating the variance from a population, which is defined as $$Var(X)=\dfrac{1}{n}\sum_i x_i^2-\mu^2$$

However, if all you have is a sample from a population, then the unbiased formula for the population variance is defined as $$Var(X)=\dfrac{1}{n-1}\sum_i (x_i-\bar{x})^2$$

Notice the dfference of $n-1$ instead of $n$ and the sample mean $\bar{x}$ rather than the population mean $μ$.

The R function var by default estimates the variance using the second formula, as that is almost always the case in statistics (dealing with a sample rather than a population).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.