How does one explain what an unbiased estimator is to a layperson?

Question

Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$.

How does one explain this to a layperson? In the past, what I have said is if you average a bunch of values of $\hat{\theta}$, as the sample size gets larger, you get a better approximation of $\theta$.

To me, this is problematic. I think what I'm actually describing here is this phenomenon of being asymptotically unbiased, rather than solely being unbiased, i.e., $$\lim_{n \to \infty}\mathbb{E}[\hat{\theta} \mid \theta] = \theta\text{,}$$ where $\hat{\theta}$ is likely dependent on $n$.

So, how does one explain what an unbiased estimator is to a layperson?

Answer 1

Technically what you're describing when you say that your estimator gets closer to the true value as the sample size grows is (as others have mentioned) consistency, or convergence of statistical estimators. This convergence can either be convergence in probability, which says that $\lim_{n \to \infty} P(|\hat{\theta}_n - \theta| > \epsilon) = 0$ for every $\epsilon > 0$, or almost sure convergence which says that $P(\lim_{n \to \infty} |\hat{\theta}_n - \theta| > \epsilon) = 0$. Notice how the limit is actually inside the probability in the second case. It turns out this latter form of convergence is stronger than the other, but both of them mean essentially the same thing, which is that the estimate tends to get closer and closer to the thing we're estimating as we gather more samples.

A subtle point here is that even when $\hat{\theta}_n \to \theta$ either in probability or almost surely, it is not true in general that $\lim_{n \to \infty} \text{E}(\hat{\theta}_n) = \theta$, so consistency does not imply asymptotic unbiasedness as you're suggesting. You have to be careful when moving between sequences of random variables (which are functions) to sequences of expectations (which are integrals).

All the technical stuff aside, unbiased only means that $\text{E}(\hat{\theta}_n) = \theta$. So when you explain it to someone just say that if the experiment were repeated under identical conditions many times that the average value of the estimate would be close to the true value.

Answer 2

I am not sure if you confuse consistency and unbiasedness.

Consistency: The larger the sample size the closer the estimate to the true value.

• Depends on sample size

Unbiasedness: The expected value of the estimator equals the true value of the parameters

• Does not depend on sample size

So your sentence

if you average a bunch of values of $\hat\theta$, as the sample size gets larger, you get a better approximation of $\theta$.

Is not correct. Even if the sample size gets infinite an unbiased estimator will stay an unbiased estimator, e.g. If you estimate the mean as "mean +1" you can add one billion observations to your sample and your estimator will still not give you the true value.

Here you can find a more profound discussion about the difference between consistency and unbiasedness.

What is the difference between a consistent estimator and an unbiased estimator?

Answer 3

@Ferdi already provided clear answer to your question, but let's make it a little bit more formal.

Let $X_1,\dots,X_n$ be your sample of independent and identically distributed random variables from distribution $F$. You are interested in estimating unknown but fixed quantity $\theta$, using estimator $g$ being a function of $X_1,\dots,X_n$. Since $g$ is a function of random variables, estimate

$$ \hat\theta_n = g(X_1,\dots,X_n)$$

is also a random variable. We define bias as

$$ \mathrm{bias}(\hat\theta_n) = \mathbb{E}_\theta(\hat\theta_n) - \theta $$

estimator is unbiased when $\mathbb{E}_\theta(\hat\theta_n) = \theta$.

Saying it in plain English: we are dealing with random variables, so unless it's degenerate, if we took different samples, we could expect to observe different data and so different estimates. Nonetheless, we could expect that across different samples "on average" estimated $\hat\theta_n$ would be "right" if the estimator is unbiased. So it would not be always right, but "on average" it would be right. It simply cannot always be "right" because of randomness associated with the data.

As others already noted, the fact that your estimate gets "closer" to estimated quantity as your sample grows, i.e. that in converges in probability

$$ \hat\theta_n \overset{P}{\to} \theta $$

has to do with estimators consistency, not unbiasedness. Unbiasedness alone does not tell us anything about sample size and its relation to obtained estimates. Moreover, unbiased estimators are not always available and not always preferable over biased ones. For example, after considering bias-variance tradeoff you may be willing to consider using estimator with greater bias, but smaller variance -- so "on average" it would be farther from the true value, but more often (smaller variance) the estimates would be closer to the true value, then in case of unbiased estimator.

Answer 4

First you must distinguish misunderstanding bias from statistical bias, especially for a lay person.

The choice of say using median, mean or mode as your estimator for a population average, often contains a political, religious or science theory belief bias. The computation as to which estimator is the best form of average is of a different type to the arithmetic that affects statistical bias.

Once you have got past the method selection bias, then you can address the potential biases in the estimation method. First you have to pick a method that can have a bias, and a mechanism that leads easily toward that bias.

It can be easier to use a divide a conquer viewpoint where it becomes obvious as the sample size gets smaller, the estimate becomes clearly biased. For example the n-1 factor (vs 'n' factor) in sample spread estimators becomes obvious as n drops from 3 to 2 to 1 !

It all depends on how 'lay' the person is.