# Are all estimators biased? Is the unbiasedness only a theoretical or approximation case?

The definition of unbiased estimator says that it's expected value has no difference comparing to a true value. So can we say that all estimators are biased (even slightly)? I thought that only in terms of infinite sampling we can converge to a true value, hence in this conditions estimators are not biased. Is the term unbiased estimator just a generalization or some real scenario issue? Why I see many times that people are calling estimators used in a statistical inference - unbiased. Is it just a approximation isue? What about convergence to a true value and infinite variance of population? Any help would highly appreciated!

As you said, unbiasedness is a theoretical property of an estimator because the expected value operation is theoretical, which means it doesn't depend on the sample size. So, an estimator is either biased or unbiased and doesn't change its state according to $$n$$.

I thought that only in terms of infinite sampling we can converge to a true value

I'd also like to comment on this one:

For example, let $$\hat \theta_n$$ be the n-sample estimator of parameter $$\theta$$. If $$\mathbb E[\hat \theta_n]=\theta$$, the estimator is unbiased and this means

• as $$n$$ increases, you get close to true $$\theta$$
• when averaged among all n-sample estimates, the estimator will converge to the true $$\theta$$ (this statement is the implication of the expected value)

Note that, the first one can also be satisfied by unbiased estimators, but the second cannot. Take for example, $$\mathbb E[\hat\theta_n]=\frac{n}{n-1}\theta$$. Then,

$$\lim_{n\rightarrow\infty}\mathbb E[\hat\theta_n]=\lim_{n\rightarrow\infty}\frac{n}{n-1}\theta=\theta$$

So, as the sample size, $$n$$, increases, the unbiased estimator converges to the true parameter.

• Thank you for the response. So can ever the estimator's expected value meet the "true" true value (except when the sample is complete)? Or we just call the estimator unbiased when it's arbitraly close to the true value? For example does the BLUE estimator really gives us the "true" true value or it just close enough to the true value so we can call it unbiased? – Tom Apr 8 at 12:06
• an unbiased estimator's expected value is always equal to the true parameter, however you won't be able to measure it because $\mathbb E[.]$ is not an operation performed over the data. It's purely theoretical. – gunes Apr 8 at 12:14
• So, an estimator is either biased or unbiased and doesn't change its state according to n This seems very strong, might be true in some cases, but certainly the bias can depend on $n$. – kjetil b halvorsen Apr 8 at 14:08
• Indeed, @kjetilbhalvorsen , the bias in my second example is a function of $n$. – gunes Apr 8 at 14:16