# Estimator bias versus sampling bias

When teaching a course, what is the best way to explain/call sampling bias? I don't know any synonyms for this and it seems the consensus on here is that the word bias should be reserved for the statistical bias of an estimator.

• What about "sampling bias"? I'm pretty sure that if you asked social scientists, there would be consensus that bias is sampling bias. Language is ambiguous. Moreover, even if you used the term consistently, your students will see "sampling bias" all over the literature, so you won't guard them against it, so maybe better just acknowledge that there are two kinds of biases? – Tim Aug 13 '18 at 13:51
• I see, thanks for your insight. I guess I was hoping there was a distinct way to call these phenomena to avoid confusion. – Frans Rodenburg Aug 13 '18 at 14:15

Maybe some simple examples of 'sampling bias' and of 'estimator bias' would help your students understand the difference:

Sample bias: A convenience sample of people chosen in a shopping district and a convenience sample of people chosen in a financial district might show very different views about tax policy. Getting 'unbiased' opinions of the general population or of people registered to vote is much more difficult. It requires choosing subjects at random from the desired population and (because not everyone will want to participate) finding a way to correct for non-responses.

If you want opinions of mothers with children in school, you can't choose school children at random and then ask their mothers. Mothers with several children in school will be over-represented.

Estimator bias. This is a less intuitive and more technical matter. A simple example is estimating the parameter $\theta$ using data from the distribution $\mathsf{Unif}(0, \theta).$ The maximum likelihood estimator is the maximum $X_{(n)}$ of the $n$ observations in the sample, but the maximum must logically be less than $\theta$ and consequently biased. In fact, the mean of $X_{(n)}$ is $E(X_{(n)})=\frac{n}{n+1}\theta,$ so an unbiased estimator is $\frac{n+1}{n}X_{(n)},$ and correction for bias is easy.

A related discrete example that may be of interest to an elementary class is the 'German tank problem', which you can google.

A biased estimator in common use is the sample standard deviation $S = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (X_i - \bar X)^2}$ as an estimate of the population standard deviation $\sigma$ of normal data. The bias is small for large $n,$ but noticeably large for very small $n.$ A simple simulation in R statistical software can illustrate this for $n = 4$ and $\sigma = 10$ by looking at 1000 samples of size four from $\mathsf{Norm}(\mu=0, \sigma=10).$

set.seed(813);  m = 1000;  n = 4;  sg = 10
s = replicate(m, sd(rnorm(n, 0, sg)))
mean(s)
[1] 9.185391


With only a thousand iterations one can't expect good accuracy. But several additional simulations (without the set.seed statement, hence seeds chosen unpredictably by the computer) gave results noticeably below 10: 9.351062, 9.283251, 9.006413, and so on. The exact expectation for $n \ge 2$ is $E(S_n) = \sigma\sqrt{\frac{2}{n-1}}\Gamma(\frac{n}{2})/\Gamma(\frac{n-1}{2}).$ Thus $E(S_4) = 9.213177.$ [Of course, the sample variance is unbiased: $E(S_n^2) = \sigma^2.]$

10*sqrt(2/3)*gamma(2)/gamma(1.5)
[1] 9.213177


I tried several runs with $100$ samples of four (which might be feasible for a class project using calculators) without getting any mean of 100 $S_4$'s above $10.$

Because few samples in practical applications have $n$'s small enough for the bias of $S$ to be a difficulty, most statisticians don't bother to use unbiased estimators. However, in quality management, control charts often use unbiased estimators of $\sigma$ based on the sample range, which work well for very small samples (but not for larger samples).

• This is a nice example, thanks for writing that out! – Frans Rodenburg Aug 14 '18 at 8:34