Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
This is more of a history question than a technical question.
Why is the ``Neyman-Pearson lemma'' a Lemma and not a Theorem?
link to wiki: https://en.wikipedia.org/wiki/Neyman%E2%80%93Pearson_lemma
NB: The question is not about what is a lemma and how lemmas are used to prove a theorem, but about the history of the Neyman-Pearson lemma. Was it used to prove a theorem and then it happened to be more useful? Is there any evidence of this beyond suspicion that this was the case?
NB: This historically first answer to the OP question. In statistics, the Neyman–Pearson lemma was introduced by Jerzy Neyman and Egon Pearson in a paper in 1933.. Also, it is used in practice by statisticians as a theorem, not a lemma, and it is called a lemma largely because of the 1936 paper. IMHO, the historical treatment does not answer the "why" question, and this post attempts to do that.
Lemmas are what people refer to as lemmas, and not some fixed idea of what a lemma should be. In that light, the Neyman–Pearson lemma is a lemma because that is what it is called.
What a lemma is as contrasted to a theorem or corollary is addressed elsewhere and here. More exactly, as to the matter of definition: Lemma, first meaning: A subsidiary or intermediate theorem in an argument or proof. I agree with the Oxford dictionary but would have changed the word order, and note the exact language: intermediate or subsidiary theorem. Some authors believe that a lemma must be intermediary in a proof, and this is the case for many unnamed lemmas. However, it is common, at least for named lemmas, for the lemma result to be an implication arising from an already proven theorem such that the lemma is an additional, i.e., subsidiary theorem. From the New World Encyclopedia The distinction between theorems and lemmas is rather arbitrary, since one mathematician's major result is another's minor claim. Gauss' lemma and Zorn's lemma, for example, are interesting enough per se that some authors present the nominal lemma without going on to use it in the proof of any theorem. Another example of this is Evans lemma, which follows not from proof of a simple theorem of differential geometry which...shows that the first Cartan structure equation is an equality of two tetrad postulates...The tetrad postulate [Sic, itself] is the source of the Evans Lemma of differential geometry. Wikipedia mentions the evolution of lemmas in time: In some cases, as the relative importance of different theorems becomes more clear, what was once considered a lemma is now considered a theorem, though the word "lemma" remains in the name.
However, note well that whether or not they stand alone lemmas are also theorems. That is, a theorem that is a lemmas may sometimes be an answer to the question, "What does the (above) theorem imply?" Sometimes lemmas are a stepping stone used to establish a theorem.
It is clear from reading the 1933 paper: IX. On the problem of the most efficient tests of statistical hypotheses. Jerzy Neyman, Egon Sharpe Pearson, and Karl Pearson, that the theorem being explored is Bayes' theorem. Some readers of this post have difficulty relating Bayes' theorem to the 1933 paper despite an introduction that is rather explicit in that regard. Note that the 1933 paper is littered with Venn diagrams, Venn diagrams illustrate conditional probability, which is Bayes' theorem. Some people refer to this as Bayes' rule, as it is an exaggeration to refer to that rule as being a "theorem." For example, if we were to call 'addition' a theorem, as opposed to being a rule, we would confound rather than explain.
Therefore, the Neyman-Pearson lemma is a theorem concerning the most efficient testing of Bayesian hypotheses, but is not currently called that because it was not to begin with.
The classic version appears in 1933, but the earliest occasion of its being referred to as a "lemma" is possibly in Neyman and Pearson's 1936 article Contributions to the theory of testing statistical hypotheses (pp. 1-37 of Statistical Research Memoirs Volume I). The lemma, and the proposition it was used to prove, were stated as follows:
This is known today as the generalized Neyman-Pearson Fundamental Lemma (cf. Chapter 3.6 of Lehman and Romano's Testing Statistical Hypotheses), and it reduces to your everyday Neyman-Pearson when $m=1$. The lemma itself was then studied by several big names from that era (e.g. P.L. Hsu, Dantzig, Wald, Chernoff, Scheffé) and the name "Neyman and Pearson's lemma" thus stuck.
Here's a list of relevant articles/books if one's interested in the history of the Neyman-Pearson lemma:
