# What does "unbiasedness" mean?

• What does it mean to say that "the variance is a biased estimator".
• What does it mean to convert a biased estimate to an unbiased estimate through a simple formula. What does this conversion do exactly?
• Also, What is the practical use of this conversion? Do you convert these scores when using certain kind of statistics?

You can find everything here. However, here is a brief answer.

Let $\mu$ and $\sigma^2$ be the mean and the variance of interest; you wish to estimate $\sigma^2$ based on a sample of size $n$.

Now, let us say you use the following estimator:

$S^2 = \frac{1}{n} \sum_{i=1}^n (X_{i} - \bar{X})^2$,

where $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$ is the estimator of $\mu$.

It is not too difficult (see footnote) to see that $E[S^2] = \frac{n-1}{n}\sigma^2$.

Since $E[S^2] \neq \sigma^2$, the estimator $S^2$ is said to be biased.

But, observe that $E[\frac{n}{n-1} S^2] = \sigma^2$. Therefore $\tilde{S}^2 = \frac{n}{n-1} S^2$ is an unbiased estimator of $\sigma^2$.

Footnote

Start by writing $(X_i - \bar{X})^2 = ((X_i - \mu) + (\mu - \bar{X}))^2$ and then expand the product...

The expected value of $S^2$ does not give $\sigma^2$ (and hence $S^2$ is biased) but it turns out you can transform $S^2$ into $\tilde{S}^2$ so that the expectation does give $\sigma^2$.

In practice, one often prefers to work with $\tilde{S}^2$ instead of $S^2$. But, if $n$ is large enough, this is not a big issue since $\frac{n}{n-1} \approx 1$.

Remark Note that unbiasedness is a property of an estimator, not of an expectation as you wrote.

• I mean more in theoretical terms. I can find the formula in any book, but i'm interested more in the explanation in words. The expectation of the sigma is unbiased and we can transform the estimate into the expectation? Sep 24 '11 at 9:47
• also i'm asking about the practical aspects of this, do you use this conversion while performing analyses? Sep 24 '11 at 9:47
• @ocram What is $n$? Is it the sample size? Or number of samples taken? Or both? Dec 25 '16 at 17:22
• @quirik: The assumption is that a single sample is taken and that this sample is of size n Dec 25 '16 at 17:36
• @ocram How do we then calculate expected value of variance if we have one sample? What am I missing? Dec 25 '16 at 17:39

This response clarifies ocram's answer. The key reason (and common misunderstanding) for $E[S^2] \neq \sigma^2$ is that $S^2$ uses the estimate $\bar{X}$ which is itself estimated from data.

If you work through the derivation, you will see that the variance of this estimate $E[(\bar{X}-\mu)^2]$ is exactly what gives the additional $-\frac{\sigma^2}{n}$ term

The explanation that @Ocram gave is great. To explain what he said in words: if we calculate $s^2$ by dividing just by $n$, (which is intuitive) our estimation of $s^2$ will be an underestimate. To compensate, we divide by $n-1$.

Here's an exercise: Make up a discrete probability with 2 outcomes, say $P(2) = .25$ and $P(6) = .75$. Find $\mu$ and $\sigma$ for this distribution. Calculate $\mu$ and $\sigma$ for the sample mean when $n = 3$. Calculate all possible samples of size $n =3$. Calculate $s^2$ over those samples, and apply appropriate frequencies.

Sometimes, you gotta get your hands dirty.

• thanks for your help. A few questions: In your excercise: what kind of distribution are you referring to, Binomial? What do you mean make up a discrete probability? You mean calculate all the probabilities of 2 and 6 over different sample sizes? Sep 26 '11 at 20:46
• They mean a distribution over a binary variable whose only two values are 2 and 6, the first with probability .25 and the second with probability 0.75. I don't know why they chose values 2 and 6 -- probably to stress that the values are not really relevant; it would have been simpler to talk a Bernoulli-distributed variable with probability of success 0.25, and things would work the same. Sep 20 at 23:15

Generally using "n" in the denominator gives smaller values than the population variance which is what we want to estimate. This especially happens if the small samples are taken. In the language of statistics, we say that the sample variance provides a “biased” estimate of the population variance and needs to be made "unbiased".