9
$\begingroup$

I am very interested in the historical development of statistical theories. Here is the research I've done: I've tried to read two old papers of Fisher. I think the first theory paper on MLE should be Fisher 1912.

Also, Thurstone had some models to construct a probability mass function which is "equivalent" to a likelihood function, but from two old papers of him, I cannot confirm that he mentioned the maximum likelihood estimation. He implemented his idea but I don't know if he had done something similar to an MLE estimation of the parameterized function.

So my question is, what is the paper on implementing MLE in practice, especially the discrete data set? Who invented the probability mass function/likelihood function? Because Fisher invented MLE, do people usually attribute the invention of likelihood function to him?

Improve this question
$\endgroup$
4
  • 6
    $\begingroup$ Your question is not sufficiently specific. The earliest MLE is arguably the first time someone computed a mean using a normal model ... so maybe de Moivre? I think you could reasonably argue that the earliest attempt to deliberately set out to maximize likelihood would probably be Gauss though of course he didn't call it likelihood nor develop use of MLEs as a general theory. The earliest attempts to maximize likelihood they way we look at it now would presumably be Fisher, though probability mass functions are far, far older. Arguably de Moivre does so in The Doctrine of Chances ... ctd $\endgroup$
    – Glen_b
    Commented Dec 27, 2019 at 7:22
  • 2
    $\begingroup$ ... (though not necessarily in a way you'd immediately recognize); he certainly writes out a binomial expansion where the terms in the sum each represent binomial probabilties; I'd argue he's giving a form of the pmf when doing so (he also develops the normal approximation to the binomial in the same document). So I think pmfs are at least that old. As a result you may need to revise your premises (like that pmfs and likelihood come up at the same time) and to be clearer about what you seek in an answer. $\endgroup$
    – Glen_b
    Commented Dec 27, 2019 at 7:24
  • 3
    $\begingroup$ I don't think it's really necessary to include both likelihood and maximum likelihood tags; you could choose whichever is most appropriate - perhaps maximum likelihood - which would then leave room for history $\endgroup$
    – Glen_b
    Commented Dec 27, 2019 at 7:30
  • 2
    $\begingroup$ Look at Anders Hald's book A History of Parametric Statistical Inference from Bernoulli to Fisher, 1713-1935 He will give reasonably modern uses before Fisher. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Dec 27, 2019 at 13:13

1 Answer 1

Reset to default
12
$\begingroup$

The most relevant reference imho is Steve Stigler's "Epic history of maximum likelihood" (2007)

"There were early intelligent comments related to this problem [of seeking the most probable distribution for the observation] already in the 1750s by Thomases Simpson and Bayes and by Johann Heinrich Lambert in 1760, but the first serious assault related to our topic was by Joseph Louis Lagrange in 1769." S. Stigler (2007)

"By introducing restrictions in the form of the curve only after deriving the estimates of probabilities, Lagrange’s analysis had the curious consequence of always arriving at method of moment estimates, even though starting with maximum likelihood!" S. Stigler (2007)

He also points out at Daniel Bernoulli (1769) and Carl Friedrich Gauß (1809), albeit the later started using Bayesian arguments to see the MLE as a posterior mode.

"...a long memoir by Karl Pearson and Louis Napoléon George Filon, published in the Transactions of the Royal Society of London in 1898 has a place in history, more for what in the end it seemed to suggest, rather than for what it accomplished." S. Stiegler (2007)

"...the method of maximum likelihood was proposed independently by Lambert and Daniel Bernoulli, but with no practical effect because the maximum likelihood equation for the error distribution considered was intractable." A. Hald (1999)

"It is an astounding fact that Edgeworth’s papers were unknown to Fisher when he wrote his paper on maximum likelihood estimation in 1912." A. Hald (1999)

A. Hald (1999) also mentions Encke (1832) and Hagen (1837) as maximising $p(\mathbf x|\theta)$ in $\theta$ to find the "most probable" sample. He further cites Chauvenet (1863) and Merriman (1884) before Edgeworth (1908).

"Edgeworth (1908) anticipated a good part of the (Fisher) 1922 version, but nobody noticed until a decade or so after Fisher had redone it." J. Aldrich (1997)

"...the [maximum likelihood] criterion appears at the head of the derivation of least squares in Chauvenet (1891, p.481), Bennett (1908, p.15) and Brunt (1917,p.77)" J. Aldrich (1997)

" Pearson (1896, p.265) states that the "best" value of r is found by choosing the value for which "the observed result is the most probable." J. Aldrich (1997)

Looking at Thurstone's bibliography, it does not appear a relevant paper predates 1912.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thank you for your wisdom. So in sum, Lambert and Daniel Bernoulli were the first people who proposed MLE, but Fishier was first one who put MLE in successful practical use? $\endgroup$
    – High GPA
    Commented Jul 5, 2020 at 4:29
  • $\begingroup$ I merely quoted from these three historical articles in Statistical Science. $\endgroup$
    – Xi'an
    Commented Jul 5, 2020 at 19:37
  • $\begingroup$ That is very rigorous and cautious. Do you have an opinion on who first proposed MLE and who first implemented it? $\endgroup$
    – High GPA
    Commented Jul 7, 2020 at 3:45
  • $\begingroup$ I have no opinion and prefer to refer to those of experts like Steve Stigler. Given the early confusion between Bayesian and non-Bayesian methods, it is further difficult to be sure that earlier proposals were truly seen as maximum likelihood estimators. $\endgroup$
    – Xi'an
    Commented Jul 8, 2020 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.