# What is "Likelihood Principle"?

While I was studying "Bayesian Inference", I happen to encounter the term, "Likelihood Principle" but I don't really get the meaning of it. I assume it is connected to "Bayesian Inference" but I am not really sure what it is and what part of "Bayesian Inference" it is connect to and what it does.

I have read the "Example" part in WikiPedia for "Likelihood Principle" but it is not clear that why the second case coefficient of the binomial distribution is different and I hope to get the translation of this part which is kind of unclear to me.

• In my humble opinion, the likelihood principle is simply wrong. It proposes that all the evidence relevant for the test is in the likelihood function. But in reality, data only become evidence for something by means of a proper testing procedure. The evidence is clearly NOT solely in the data, but in the data AND the procedure. (Much like circumstantial evidence has to be evaluated in a court of law, by the way.) You should be very skeptical if somebody suggests that by using Bayesian or likelihood statistics, you could rid yourself of the classical problems of multiple testing, sequential sam May 3 at 15:18

## 1 Answer

The concept of the likelihood principle (LP) is that the entire inference should be based on the likelihood function and solely on the likelihood function. Informally, the likelihood function is sufficient for conducting inference, meaning that the sampling model and the sample itself can be ignored once the likelihood function is constructed.

The example used in the Wikipedia page is traditional: if the result of an opinion poll is that, out of 12 persons, 3 favour candidate T (and 8 candidate B), two possible sampling models (among an infinity of compatible models!) agree with this observation:

1. The poll has been conducted with a fixed sample of size $$n=12$$ and $$X=3$$ of the 12 persons came as supporting candidate T. This means the sampling distribution is a Binomial $$\mathcal {Bin}(12,\theta)$$ and $$X=3$$ is the observed realisation from this model;

2. The poll has been conducted with a goal of $$m=3$$ supporters of candidate T and people have been interviewed until $$m=3$$ were favouring T. This means the sampling distribution is a Negative Binomial $$\mathcal {Neg}(3,\theta)$$ and $$X=12$$ is the observed realisation from this model.

The respective likelihoods are $$\ell_1(\theta) = {12 \choose 9} \theta^3(1-\theta)^9\qquad \ell_2(\theta) = {11 \choose 9} \theta^3(1-\theta)^9$$ and, as functions of $$\theta$$, they are proportional. The Likelihood Principle then states that inference should be the same in both cases, despite the distribution of the sample differing between the two modellings.

Besides agreeing with most of Bayesian inference, but not all of it, as e.g. Jeffreys' priors, it also has serious impacts on other branches of statistical inference. It is usually derived as a consequence of the sufficiency principle and of the conditioning principle (Allan Birnbaum, JASA, 1962). The book

The likelihood principle: a review, generalizations, and statistical implications by James O. Berger and Robert L. Wolpert

is available for free on Project Euclid and provides an in-depth study of the principle.

You can also check the discussion paper by science philosopher Deborah Mayo

On the Birnbaum Argument for the Strong Likelihood Principle

also available for free on Project Euclid, which disagrees with its derivation (and its relevance). She argues that the strong form of the Likelihood Principle cannot be derived from the two other principles.

See also this answer to an earlier X validated question on the Likelihood Principle.

• I can understand why the LP is controversial. But I can’t understand why Mayo’s objections to the derivation haven’t been swiftly resolved one way or another. What’s going on here? Oct 15, 2020 at 12:42
• If you read the Statistical Science paper and its discussions, you will see both sides of the coin. Oct 15, 2020 at 13:36
• Thanks, I found this useful stats.stackexchange.com/questions/379798/… Oct 16, 2020 at 0:07
• 9 favouriting candidate B? Nov 6, 2020 at 9:07
• @ChristophHanck: yes indeed..! Nov 6, 2020 at 9:21