# CLT and Biased Estimators

A biased estimator is one for which the difference of the expected value of the estimator and the true value of a population parameter does not equal zero. If we gather a bunch of samples' averages (countably many) and take the average of that collection of samples, the mean should equal the true value of the parameter if those sample averages were unbiased.

On the other hand, I've seen some resources state that the mean of a sampling distribution (i.e. a set of sample means) should equal the population parameter, regardless of the estimator used to calculate the sample means.

I can't reconcile these two statements. Can y'all?

Your explanation of bias is correct, so long as you understand that bias is always measured with respect to a parameter you are estimating. Without further assumptions, it is not generally true that the sample mean is an unbiased estimator of "the population parameter" (presumably you mean some mean parameter applicable to the population, but you would need to specify exactly what this parameter is).

Though the latter property is not true in general, it is true in a broad class of cases. In particular, if you have a sequence of observable random variables $X_1, X_2, X_3, ...$ that have a common mean parameter $\mu = \mathbb{E}(X_i)$ then you have:

$$\mathbb{E}(\bar{X}_n) = \mathbb{E} \Big( \frac{1}{n} \sum_{i=1}^n X_i \Big) = \frac{1}{n} \sum_{i=1}^n \mathbb{E}(X_i) = \frac{1}{n} \sum_{i=1}^n \mu = \mu.$$

In this case the sample mean is an unbiased estimator of the mean parameter $\mu$. This occurs in the case of IID data (i.e., data drawn via simple random sampling), but it also occurs more broadly, if there is not a common distribution, but only a common mean. Oh, and incidentally, this question relates only to the properties of moments - it has nothing to do with the CLT.

I think your confusion is in the phrasing.

A estimator $\hat \beta$ is biased for a parameter $\beta$ if $E[\hat \beta] \not = \beta$. Important is the fact that an estimator is biased in comparison with what you want to estimate.

Example: if a skewed probability distribution $X$ has expected value $\mu$ then $\overline X_n$ (the mean of sample sized $n$) is a unbiased estimator for $\mu$. But you could also use something else to estimate $\mu$, say $T_n =$ the trimmed or truncated 90% mean (remove 5% at the left, 5% at the right and take the mean of the remaining 90%). Since the distribution is skewed $T_n$ will be biased in respect to $E[X] = \mu$.

Using other (biased) estimators can be interesting because they might have other more desirable properties (like ease of calculation, or less variability, ...

You already know that given a certain skewed distribution $X$ then the CLT guarantees $\sqrt{n}(\overline X_n - \mu)\stackrel{\text{d}}{\rightarrow} N\left(0,\sigma^2\right)$
You should think about this as follows. Conduct the experiment $n$-times, and calculate the sample mean each time. Now consider those sample means, they will convergence towards a normal distribution. (With mean $\mu$ and the variance of these sample means will approach zero)
The same would be valid for the truncated mean, it would also converge to a normal distribution, but not the same one as the sample mean. The expected value of these $T_n$ would not equal $\mu$ (as it is biased).
• @LumpyGrads, note that an estimator need not converge to a normal distribution. For example, instead of the mean $\overline X_n,$ suppose we throw away all the data in an iid sample except the last observation, $X_n$, and we use this as an estimator for the expected value $\mu$. Clearly we cannot expect $\sqrt{n}(X_n - \mu)$ to converge in distribution to a normal distribution. For an estimator $\hat \mu$, the special circumstance where $\sqrt{n}(\hat \mu - \mu)\stackrel{\text{d}}{\rightarrow} N\left(0,\sigma^2\right)$ is called asymptotic normality; it is not true for all estimators. – Breaking Waves Jul 31 '18 at 21:35