11
$\begingroup$

This is more of a history question than a technical question.

Why is the ``Neyman-Pearson lemma'' a Lemma and not a Theorem?

link to wiki: https://en.wikipedia.org/wiki/Neyman%E2%80%93Pearson_lemma

NB: The question is not about what is a lemma and how lemmas are used to prove a theorem, but about the history of the Neyman-Pearson lemma. Was it used to prove a theorem and then it happened to be more useful? Is there any evidence of this beyond suspicion that this was the case?

$\endgroup$
3
  • 6
    $\begingroup$ Terminology: A lemma is a "helping theorem", a proposition with little applicability except that it forms part of the proof of a larger theorem. In some cases, as the relative importance of different theorems becomes more clear, what was once considered a lemma is now considered a theorem, though the word "lemma" remains in the name. $\endgroup$
    – Carl
    Dec 29, 2018 at 2:42
  • 2
    $\begingroup$ @Carl Sure, but why is the Neyman-Pearson lemma a lemma and not a theorem? was there a Theorem? and is there evidence of it? As I said, it is history question, not a technical one. $\endgroup$
    – Tauto
    Dec 29, 2018 at 3:12
  • 2
    $\begingroup$ Well, the N-P lemma's used to prove the Karlin-Rubin theorem, & that Rao's score test is locally most powerful; these results are perhaps applied more widely than the N-P lemma itself (point null vs point alternative). $\endgroup$ Jan 2, 2019 at 12:49

2 Answers 2

0
$\begingroup$

NB: This historically first answer to the OP question. In statistics, the Neyman–Pearson lemma was introduced by Jerzy Neyman and Egon Pearson in a paper in 1933.. Also, it is used in practice by statisticians as a theorem, not a lemma, and it is called a lemma largely because of the 1936 paper. IMHO, the historical treatment does not answer the "why" question, and this post attempts to do that.

What a lemma is as contrasted to a theorem or corollary is addressed elsewhere and here. More exactly, as to the matter of definition: Lemma, first meaning: A subsidiary or intermediate theorem in an argument or proof. I agree with the Oxford dictionary but would have changed the word order, and note the exact language: intermediate or subsidiary theorem. Some authors mistakenly believe that a lemma must be intermediary in a proof, and this is the case for many unnamed lemmas. However, it is common, at least for named lemmas, for the lemma result to be an implication arising from an already proven theorem such that the lemma is an additional, i.e., subsidiary theorem. From the New World Encyclopedia The distinction between theorems and lemmas is rather arbitrary, since one mathematician's major result is another's minor claim. Gauss' lemma and Zorn's lemma, for example, are interesting enough per se that some authors present the nominal lemma without going on to use it in the proof of any theorem. Another example of this is Evans lemma, which follows not from proof of a simple theorem of differential geometry which...shows that the first Cartan structure equation is an equality of two tetrad postulates...The tetrad postulate [Sic, itself] is the source of the Evans Lemma of differential geometry. Wikipedia mentions the evolution of lemmas in time: In some cases, as the relative importance of different theorems becomes more clear, what was once considered a lemma is now considered a theorem, though the word "lemma" remains in the name.

However, note well that whether or not they stand alone lemmas are also theorems. That is, a theorem that is a lemmas may sometimes be an answer to the question, "What does the (above) theorem imply?" Sometimes lemmas are a stepping stone used to establish a theorem.

It is clear from reading the 1933 paper: IX. On the problem of the most efficient tests of statistical hypotheses. Jerzy Neyman, Egon Sharpe Pearson, and Karl Pearson, that the theorem being explored is Bayes' theorem. Some readers of this post have difficulty relating Bayes' theorem to the 1933 paper despite an introduction that is rather explicit in that regard. Note that the 1933 paper is littered with Venn diagrams, Venn diagrams illustrate conditional probability, which is Bayes' theorem. Some people refer to this as Bayes' rule, as it is an exaggeration to refer to that rule as being a "theorem." For example, if we were to call 'addition' a theorem, as opposed to being a rule, we would confound rather than explain.

Therefore, the Neyman-Pearson lemma is a theorem concerning the most efficient testing of Bayesian hypotheses, but is not currently called that because it was not to begin with.

$\endgroup$
42
  • 2
    $\begingroup$ I'm a bit confused as to what exactly you're saying here. Clearly not that the N-P lemma is used to prove Bayes' theorem, in this paper or elsewhere. So the question "Why 'lemma'?" remains. The N-P lemma is used in Sections III & IV of this paper in the derivation of UMP similar tests, & might justly have been called a lemma for this reason. $\endgroup$ Jan 4, 2019 at 9:59
  • 4
    $\begingroup$ Your statement "Therefore, the Neyman-Pearson lemma could be called a theorem " is unfounded and explains nothing why we refer to the 'Neyman-Pearson lemma' as a lemma. Furthermore, what it has to do with Bayes theorem is entirely unclear and seems false. Your answer deserves downvotes for being vague and nonsensical but since you do not like those downvotes I will just state that it does deserves them without giving any. $\endgroup$ Jan 4, 2019 at 19:41
  • 2
    $\begingroup$ A lemma is just a theorem (only placed in a different context as a 'help' in a larger proof). This is not the question and it has been answered in several threads on the mathematics site. We know that lemma's can start to live a life on their own (without their former theorem's that they helped). The question explicitly asks for the history of this in relation to the Neyman Pearson Lemma. Francis has already given a fine answer to this and there is no need for another answer. I criticized your answer because it is confusing (with stuff about Bayes rule) and not helpful or even detrimental. $\endgroup$ Jan 5, 2019 at 9:50
  • 4
    $\begingroup$ Do you have a source for that interpretation/usage of the word 'lemma'? Otherwise I believe you have simply misunderstood what the 'lemma' means. To borrow language from the linked answer from the companion site, I would interpret both the current and the previous versions of this question to mean "What is the more significant result for which the Neyman-Pearsion lemma was a 'helper' fact". $\endgroup$ Jan 5, 2019 at 18:58
  • 2
    $\begingroup$ "That is an exaggeration because it does not have to be "another". " where does this claim come from? This (without being, originally, part of a proof for 'another' theorem) is not how mathematicians use the term lemma. It is very similar to the use in logic A -> B -> C and the question asks what is C in the case of the lemma B being the Neyman Pearson lemma (it is definitely not Bayes rule/theorem). $\endgroup$ Jan 5, 2019 at 20:38
10
$\begingroup$

The classic version appears in 1933, but the earliest occasion of its being referred to as a "lemma" is possibly in Neyman and Pearson's 1936 article Contributions to the theory of testing statistical hypotheses (pp. 1-37 of Statistical Research Memoirs Volume I). The lemma, and the proposition it was used to prove, were stated as follows: enter image description here

This is known today as the generalized Neyman-Pearson Fundamental Lemma (cf. Chapter 3.6 of Lehman and Romano's Testing Statistical Hypotheses), and it reduces to your everyday Neyman-Pearson when $m=1$. The lemma itself was then studied by several big names from that era (e.g. P.L. Hsu, Dantzig, Wald, Chernoff, Scheffé) and the name "Neyman and Pearson's lemma" thus stuck.

Here's a list of relevant articles/books if one's interested in the history of the Neyman-Pearson lemma:

  • The Neyman–Pearson Story: 1926-34, E.S. Pearson, in Research Papers in Statistical: Festschrift for J. Neyman.
  • Introduction to Neyman and Pearson (1933) On the Problem of the Most Efficient Tests of Statistical Hypotheses, E.L. Lehmann, in Breakthroughs in Statistics: Foundations and Basic Theory.
  • Neyman-From Life, C. Reid.
$\endgroup$
11
  • $\begingroup$ Yes, but, Neyman-Pearson's lemma fit the definition a lemma in 1933, i.e., it was a lemma at that time, which is why it was subsequently referred to as a lemma. $\endgroup$
    – Carl
    Jan 3, 2019 at 21:35
  • 1
    $\begingroup$ @Carl, what is your point by using 'but'. Is there something wrong with this answer? $\endgroup$ Jan 4, 2019 at 10:43
  • 1
    $\begingroup$ @MartijnWeterings: You can search the term on Google Scholar and confine the date range. The earliest use is from P.L. Hsu it seems. Wald's lecture note from 1940 also cited it. $\endgroup$
    – Francis
    Jan 4, 2019 at 16:27
  • 3
    $\begingroup$ @Carl, did you miss the following part? "NB:The question is not about what is a lemma and how lemmas are used to prove a theorem, but about the history of the Neyman-Pearson lemma." It is about the history. The question ask for context how this theorem became called a lemma. Not why a theorem (or more specifically this theorem) can be called a lemma. $\endgroup$ Jan 4, 2019 at 19:31
  • 2
    $\begingroup$ @Carl, then this answer explains nicely how it fulfilled that role and it includes some history how people have been viewing that role. $\endgroup$ Jan 4, 2019 at 19:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.