8
$\begingroup$

While I was studying "Bayesian Inference", I happen to encounter the term, "Likelihood Principle" but I don't really get the meaning of it. I assume it is connected to "Bayesian Inference" but I am not really sure what it is and what part of "Bayesian Inference" it is connect to and what it does.

I have read the "Example" part in WikiPedia for "Likelihood Principle" but it is not clear that why the second case coefficient of the binomial distribution is different and I hope to get the translation of this part which is kind of unclear to me.

$\endgroup$
12
$\begingroup$

The concept of the likelihood principle (LP) is that the entire inference should be based on the likelihood function and solely on the likelihood function. Informally, the likelihood function is sufficient for conducting inference, meaning that the sampling model and the sample itself can be ignored once the likelihood function is constructed.

The example used in the Wikipedia page is traditional: if the result of an opinion poll is that, out of 12 persons, 3 favour candidate T (and 8 candidate B), two possible sampling models (among an infinity of compatible models!) agree with this observation:

  1. The poll has been conducted with a fixed sample of size $n=12$ and $X=3$ of the 12 persons came as supporting candidate T. This means the sampling distribution is a Binomial $\mathcal {Bin}(12,\theta)$ and $X=3$ is the observed realisation from this model;

  2. The poll has been conducted with a goal of $m=3$ supporters of candidate T and people have been interviewed until $m=3$ were favouring T. This means the sampling distribution is a Negative Binomial $\mathcal {Neg}(3,\theta)$ and $X=12$ is the observed realisation from this model.

The respective likelihoods are $$\ell_1(\theta) = {12 \choose 9} \theta^3(1-\theta)^9\qquad \ell_2(\theta) = {11 \choose 9} \theta^3(1-\theta)^9$$ and, as functions of $\theta$, they are proportional. The Likelihood Principle then states that inference should be the same in both cases, despite the distribution of the sample differing between the two modellings.

Besides agreeing with most of Bayesian inference, but not all of it, as e.g. Jeffreys' priors, it also has serious impacts on other branches of statistical inference. It is usually derived as a consequence of the sufficiency principle and of the conditioning principle (Allan Birnbaum, JASA, 1962). The book

The likelihood principle: a review, generalizations, and statistical implications by James O. Berger and Robert L. Wolpert

is available for free on Project Euclid and provides an in-depth study of the principle.

You can also check the discussion paper by science philosopher Deborah Mayo

On the Birnbaum Argument for the Strong Likelihood Principle

also available for free on Project Euclid, which disagrees with its derivation (and its relevance). She argues that the strong form of the Likelihood Principle cannot be derived from the two other principles.

See also this answer to an earlier X validated question on the Likelihood Principle.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I can understand why the LP is controversial. But I can’t understand why Mayo’s objections to the derivation haven’t been swiftly resolved one way or another. What’s going on here? $\endgroup$ – innisfree Oct 15 at 12:42
  • 1
    $\begingroup$ If you read the Statistical Science paper and its discussions, you will see both sides of the coin. $\endgroup$ – Xi'an Oct 15 at 13:36
  • $\begingroup$ Thanks, I found this useful stats.stackexchange.com/questions/379798/… $\endgroup$ – innisfree Oct 16 at 0:07
  • $\begingroup$ 9 favouriting candidate B? $\endgroup$ – Christoph Hanck Nov 6 at 9:07
  • $\begingroup$ @ChristophHanck: yes indeed..! $\endgroup$ – Xi'an Nov 6 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.