18
$\begingroup$

When was the word "bias" coined to mean $\mathbb{E}[\hat{\theta}-\theta]$?

The reason why I'm thinking about this right now is because I seem to recall Jaynes, in his Probability Theory text, criticizing the use of the word "bias" used to describe this formula, and suggesting an alternative.

From Jaynes' Probability Theory, section 17.2 "Unbiased Estimators:"

Why do orthodoxians put such exaggerated emphasis on bias? We suspect that the main reason is simply that they are caught in a psychosemantic trap of their own making. When we call the quantity $(\langle\beta\rangle-\alpha)$ the 'bias', that makes it sound like something awfully reprehensible, which we must get rid of at all costs. If it had been called instead the 'component of error orthogonal to the variance', as suggested by the Pythagorean form of (17.2), it would have been clear that these two contributions to the error are on an equal footing; it is folly to decrease one at the expense of increasing the other. This is just the price one pays for choosing a technical terminology that carries an emotional load, implying value judgments; orthodoxy falls constantly into this tactical error.

$\endgroup$
  • $\begingroup$ stats.stackexchange.com/questions/207760/… You might want to check the comments here. Also, my hunch would be that Jaynes criticized it because he was a bayesian $\endgroup$ – martn Dec 1 '17 at 13:22
  • 2
    $\begingroup$ According to Favid's list of "First (?) occurrence of common terms in mathematical statistics" it was first used in 1897 although the idea goes back further than that. I suspect that is not what you wanted to know which is why is is only a comment. $\endgroup$ – mdewey Dec 1 '17 at 13:28
  • 4
    $\begingroup$ @Clarinetist it is perfectly on-topic, this is what for we have the [etymology] tag. $\endgroup$ – Tim Dec 1 '17 at 13:47
  • 2
    $\begingroup$ This is a beautiful quote from Jaynes. On a different matter, are you strictly interested in the etymology of the word "bias", or in the broader question of its historical appearance/use in statistics --- the later seems more interesting, I would say, than the strict etymology. $\endgroup$ – Carlos Cinelli Dec 2 '17 at 23:37
  • 1
    $\begingroup$ @CarlosCinelli Yeah, you're right. I mean the historical appearance - i.e., when was it coined in stats. $\endgroup$ – Clarinetist Dec 2 '17 at 23:40
8
+25
$\begingroup$

Apparently, the concept of mean bias was coined by:

Neyman, J., & Pearson, E. S. (1936). Contributions to the theory of testing statistical hypotheses. Statistical Research Memoirs, 1, 1-37.

acccording to:

Lehmann, E. L. "A General Concept of Unbiasedness" The Annals of Mathematical Statistics, vol. 22, no. 4 (Dec., 1951), pp. 587–592.

which contains a more extensive discussion on the history of this concept.

It is worth noticing that mean bias is just a type of bias, and there also exists the concept of median bias (which cannot be straightforwardly extended to the multivariate case, which may explain why it is not that popular).

$\endgroup$
  • 5
    $\begingroup$ Slight correction to the above: it appears that Neyman and Pearson defined "unbiased" in the context of hypothesis testing. It appears that, with regard to a point estimate, this concept was defined in David F. N., & Neyman, J. "Extension of the Markoff theorem on least squares," Statistical Research Memoirs, pp. 105-116. This citation is also found in the Lehmann article. $\endgroup$ – Clarinetist Dec 4 '17 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.