# What does “unbiasedness” mean?

• What does it mean to say that "the variance is a biased estimator".
• What does it mean to convert a biased estimate to an unbiased estimate through a simple formula. What does this conversion do exactly?
• Also, What is the practical use of this conversion? Do you convert these scores when using certain kind of statistics?
-

You can find everything here. However, here is a brief answer.

Let $\mu$ and $\sigma^2$ be the mean and the variance of interest; you wish to estimate $\sigma^2$ based on a sample of size $n$.

Now, let us say you use the following estimator:

$S^2 = \frac{1}{n} \sum_{i=1}^n (X_{i} - \bar{X})^2$,

where $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$ is the estimator of $\mu$.

It is not too difficult (see footnote) to see that $E[S^2] = \frac{n-1}{n}\sigma^2$.

Since $E[S^2] \neq \sigma^2$, the estimator $S^2$ is said to be biased.

But, observe that $E[\frac{n}{n-1} S^2] = \sigma^2$. Therefore $\tilde{S}^2 = \frac{n}{n-1} S^2$ is an unbiased estimator of $\sigma^2$.

Footnote

Start by writing $(X_i - \bar{X})^2 = ((X_i - \mu) + (\mu - \bar{X}))^2$ and then expand the product...

The expected value of $S^2$ does not give $\sigma^2$ (and hence $S^2$ is biased) but it turns out you can transform $S^2$ into $\tilde{S}^2$ so that the expectation does give $\sigma^2$.

In practice, one often prefers to work with $\tilde{S}^2$ instead of $S^2$. But, if $n$ is large enough, this is not a big issue since $\frac{n}{n-1} \approx 1$.

Remark Note that unbiasedness is a property of an estimator, not of an expectation as you wrote.

-
I mean more in theoretical terms. I can find the formula in any book, but i'm interested more in the explanation in words. The expectation of the sigma is unbiased and we can transform the estimate into the expectation? –  Dbr Sep 24 '11 at 9:47
also i'm asking about the practical aspects of this, do you use this conversion while performing analyses? –  Dbr Sep 24 '11 at 9:47

The explanation that @Ocram gave is great. To explain what he said in words: if we calculate $s^2$ by dividing just by $n$, (which is intuitive) our estimation of $s^2$ will be an underestimate. To compensate, we divide by $n-1$.

Here's an exercise: Make up a discrete probability with 2 outcomes, say $P(2) = .25$ and $P(6) = .75$. Find $\mu$ and $\sigma$ for this distribution. Calculate $\mu$ and $\sigma$ for the sample mean when $n = 3$. Calculate all possible samples of size $n =3$. Calculate $s^2$ over those samples, and apply appropriate frequencies.

Sometimes, you gotta get your hands dirty.

-
thanks for your help. A few questions: In your excercise: what kind of distribution are you referring to, Binomial? What do you mean make up a discrete probability? You mean calculate all the probabilities of 2 and 6 over different sample sizes? –  Dbr Sep 26 '11 at 20:46
This response clarifies ocram's answer. The key reason (and common misunderstanding) for $E[S^2] \neq \sigma^2$ is that $S^2$ uses the estimate $\bar{X}$ which is itself estimated from data.
If you work through the derivation, you will see that the variance of this estimate $E[(\bar{X}-\mu)^2]$ is exactly what gives the additional $-\frac{\sigma^2}{n}$ term