Josh Starmer says it in here.
I have been searching for a simple way to understand likelihood and it's Bayesian and Frequentist use.
Josh's way seems simple to me.
Is he correct?
$\begingroup$
$\endgroup$
7
-
2$\begingroup$ I have not looked at the links, but note that a likelihood function is not necessarily a PDF. $\endgroup$– GalenCommented Oct 16, 2023 at 21:51
-
2$\begingroup$ It depends on what exactly you mean by "a pdf." The likelihood is a function of two sets of variables -- data and parameters -- and, by definition, gives a probability value (or probability density value) for the data when the parameter values are specified. Have you looked at our highly voted threads about likelihood? $\endgroup$– whuber ♦Commented Oct 16, 2023 at 22:06
-
2$\begingroup$ I hope he didn’t use that exact phrasing, because I think a reasonable interpretation of “distribution curve” is the CDF. $\endgroup$– DaveCommented Oct 16, 2023 at 22:35
-
3$\begingroup$ Thank you for the images. The answer to "is he correct" is no: those images confuse probability density with likelihood. Please use those italicized terms to search our site for further information. $\endgroup$– whuber ♦Commented Oct 16, 2023 at 23:58
-
2$\begingroup$ Starmer has some decent material, but I’ve definitely watched some of his videos and thought, “Not exactly…” $\endgroup$– DaveCommented Oct 17, 2023 at 0:07
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
13
WHERE STARMER IS CORRECT
When you limit consideration to continuous distributions (perhaps absolutely continuous), then the claim is correct: the likelihood value relates to the y-axis value of the PDF. Note that, contrasting colloquial use, “likelihood” and “probability” differ in meaning as technical terms in statistics, as these continuous distributions have the probability of any one point equal to zero, while the y-axis values have no upper bound on how high they can go.
WHERE STARMER IS WRONG
Likelihood is a general enough idea that it is inadequate to limit consideration to the continuous case. In the discrete case, for instance, think about how high the “PDF” goes on the y-axis. There is a sense in which it goes up to infinity! Thus, the idea of considering likelihood to be the y-axis value breaks down.
It might be worth thinking about what a “PDF” would look like for a distribution that has half of the density uniform on $[0,1]$ and the other half on exactly $1/2$. It’s easy to visualize the first half as just being a straight line $y=1$ on $[0,1]$ and zero everywhere else, but how high does the “PDF” go at exactly $1/2?$
-
$\begingroup$ I am understanding from this post that likelihood can be infinite ~stats.stackexchange.com/questions/140463/… $\endgroup$– KirstenCommented Oct 17, 2023 at 21:08
-
3$\begingroup$ @Kirsten That is correct: because probability densities can become infinite, so can likelihoods. It's virtually certain, though, that any likelihood (for any reasonable model) evaluated on data will be finite. As an example, consider a Gamma$(\theta)$ model for a single observation $x.$ When $\theta \lt 1,$ the likelihood for $x=0$ is infinite. However, the chance that $x=0$ is nil. That's the distinction between probability density and probability that the video is attempting to illustrate. $\endgroup$– whuber ♦Commented Oct 17, 2023 at 22:26
-
2$\begingroup$ @Kirsten The two can coincide but do not have to. “These are sometimes the same” hardly seems like a legitimate mathematical definition. $\endgroup$– DaveCommented Oct 18, 2023 at 1:22
-
1$\begingroup$ @Kirsten There's also a philosophical diference in that the likelihood fixes a value and varies the parameters(s), while the PDF fixes the parameters and then varies the value. What this means, at least in the (absolutely) continuous case such as with a Gaussian distribution, is that the likelihood looks at the height of the PDF at, say, $x=0$ as the parameters change, such as seeing which parameter gives the highest value at $x=0$ (so $\mu=0$ for a Gaussian distribution). In contrast, the PDF would fix the parameter and then calculate how high the PDF is at $x=0$. $\endgroup$– DaveCommented Oct 23, 2023 at 20:46
-
1$\begingroup$ @Kirsten I thought he explained it quite well and even mentioned in the beginning of the video that the explanation applies to continuous distributions. What do you find to be missing from his animation? $\endgroup$– DaveCommented Oct 24, 2023 at 21:40