# Birnbaum's Theorem: Strong belief in a model $\implies$ the likelihood function must be used as a data reduction device?

Working through understanding section 6.3.2 (pg. 292-294) in Casella and Berger's Statistical Inference (2nd-ed).

The following definitions and principles are given:

Definition (Experiment): An experiment $$E$$ is the triple $$(\mathbf X,\theta,\{f(\mathbf x|\theta)\})$$ where $$X$$ is a random vector with pmf $$f(\mathbf x|\theta)$$ for some $$\theta\in\Theta$$.

"Definition" (Evidence function): $$\mathrm{Ev}(E,\mathbf x)$$ stands for the evidence about $$\theta$$ arising from $$E$$ and $$\mathbf x$$.

$$FORMAL\ SUFFICIENCY\ PRINCIPLE$$: Consider an experiment $$E=(\mathbf X,\theta,\{f(\mathbf x|\theta)\})$$ and suppose $$T(\mathbf X)$$ is a sufficient statistic for $$\theta$$. If $$\mathbf x$$ and $$\mathbf y$$ are sample points satisfying $$T(\mathbf x)=T(\mathbf y)$$, then $$\mathrm{Ev}(E,\mathbf x)=\mathrm{Ev}(E,\mathbf y)$$.

$$CONDITIONALITY\ PRINCIPLE$$: Suppose that $$E_1=(\mathbf X_1,\theta,\{f_1(\mathbf x_1|\theta)\})$$ and $$E_2=(\mathbf X_2,\theta,\{f_2(\mathbf x_2|\theta)\})$$ are two experiments, where only the unknown parameter $$\theta$$ need be common between the two experiments. Consider the mixed experiment in which the random variable $$J$$ is observed, where $$P(J=1)=P(J=2)=\tfrac{1}{2}$$ (independent of $$\theta$$, $$\mathbf X_1$$, or $$\mathbf X_2$$), and the experiment $$E_J$$ is performed. Formally the experiment performed is $$E^\ast=(\mathbf X^\ast,\theta,\{f^\ast(\mathbf x^\ast|\theta)\})$$, where $$\mathbf X^\ast=(j,\mathbf X_j)$$ and $$f^\ast(\mathbf x^\ast|\theta)=f^\ast((j,\mathbf x_j)|\theta)=\tfrac{1}{2}f_j(\mathbf x_j|\theta)$$. Then $$\mathrm{Ev}(E^\ast,(j,\mathbf x_j))=\mathrm{Ev}(E_j,\mathbf x_j).$$

$$FORMAL\ LIKELIHOOD\ PRINCIPLE$$: Suppose that we have two experiments $$E_1=(\mathbf X_1,\theta,\{f_1(\mathbf x_1|\theta)\})$$ and $$E_1=(\mathbf X_2,\theta,\{f_2(\mathbf x_2|\theta)\})$$, where the unknown parameter $$\theta$$ is the same in both experiments. Suppose $$\mathbf x_1^\ast$$ and $$\mathbf x_2^\ast$$ are sample points from $$E_1$$ and $$E_2$$, respectively, such that $$L(\theta|\mathbf x_2^\ast)=CL(\theta|\mathbf x_1^\ast)$$ for all $$\theta$$ and for some constant $$C$$ that may depend on $$\mathbf x_1^\ast$$ and $$\mathbf x_2^\ast$$ but not $$\theta$$. Then $$\mathrm{Ev}(E_1,\mathbf x_1^\ast)=\mathrm{Ev}(E_2,\mathbf x_2^\ast).$$

$$LIKELIHOOD\ PRINCIPLE\ COROLLARY$$: If $$E=(\mathbf X,\theta,\{f(\mathbf x|\theta)\})$$ is an experiment, then $$\mathrm{Ev}(E,\mathbf x)$$ should depend on $$E$$ and $$\mathbf x$$ only through $$L(\theta|\mathbf x)$$.

Theorem (Birnbaum's Theorem): The Formal Likelihood Principle follows from the Formal Sufficiency Principle and the Conditionality Principle. The converse is also true.

My understanding is one can use these principles as a guidance for constructing methods of inference in many situations, and some mathematical theory (like the Rao-Blackwell Theorem) in fact shows that in order to construct methods of inference that are in a certain sense optimal under the model, these principles has to be respected. All of these principles require strong belief in a model.

In light of the $$LIKELIHOOD\ PRINCIPLE\ COROLLARY$$, if one has strong belief in a model, must the likelihood function be used as a data reduction device to maintain consistency with these principles?

• Have a look at my answer here for what might be a very important restriction on the role of the likelihood principle in inferences: stats.stackexchange.com/questions/378454/… Commented Jul 18 at 21:11

No, a strong belief in a model does not require that we maintain consistency with all these principles. To explain the reason, we need to describe some historical background from around 1962 when Birnbaum’s proof was published.

In the pre-1962 period, Fisher had argued for a greater role for conditioning, introducing the notions of an ancillary statistic and recognizable and relevant subsets. D.R. Cox’s famous 1958 paper was also very influential, especially his mixture example. Loosely, the conditionality principle suggested by a number of examples was that we should always condition on an ancillary statistic when one exists.

Birnbaum (1962) introduced a much stronger and broader conditionality principle (SCP) due to its use of an equivalence format. He showed that the SCP and his version of the sufficiency principle implied the Likelihood Principle (LP).

Within 2 years of his proof’s publication, Birnbaum had rejected both the LP and his own conditionality principle (according to Giere 1977), although it took until 1969 for Birnbaum's rejection to appear in one of his own published articles. He rejected both principles because they were inconsistent with the frequentist confidence concept (see also his 1970 letter in Nature). For example, “If there has been ‘one rock in a shifting scene’ …. it has not been the likelihood concept …. but rather the concept by which confidence limits and hypothesis tests are usually interpreted, which we may call the confidence concept of statistical evidence. This concept is not part of the Neyman-Pearson theory of tests and confidence region estimation, which denies any role to concepts of statistical evidence, as Neyman consistently insists.” (Birnbaum 1970, p. 1033) After explaining what he means by the confidence concept having taken certain aspects from the Neyman Pearson approach, he states “The absence of a comparable property in the likelihood and Bayesian approaches is widely regarded as a decisive inadequacy” (Birnbaum 1970 p.1033). Birnbaum describes in another publication how it was his introduction of the equivalence format which gave rise to the “monster” of the Likelihood Principle.

Classical statistics soon moved on. For example, the well-known 1974 text “Theoretical Statistics” by Cox and Hinkley included a Conditionality Principle that better-captured the pre-1962 concerns about conditioning, and did not have any implications leading to a Likelihood Principle. In summary, the Likelihood Principle no longer plays a role in classical statistics, although it is still much discussed within the Likelihood and Bayesian approaches.

• I will point out that Birnbaum never formally defined his "confidence concept". I think that he was unable to do so because it is impossible to come up with anything that has the properties that he hoped it would have. If you want statistical evidence in a pure form then likelihoods are what you need. If you want "confidence" then you need to specify what flavour of confidence it is that you are after. Error rate confidence requires eschewing any detailed evaluation of statistical evidence, as Neyman maintained. Commented Jul 18 at 21:18
• You can read my detailed examination of the Birnbaum's mistaken response to an alleged counter-example here: arxiv.org/pdf/1507.08394 Commented Jul 18 at 21:20
• @MichaelLew While you are correct to say that Birnbaum's confidence concept is not formally defined, it is clear that it includes the way "confidence limits and hypothesis tests are usually interpreted", and this automatically rules out the Likelihood Principle. Moreover, it is also very clear why he rejected his own version of the Conditionality Principle: the introduction of the equivalence format was not justified by the examples which had concerned Fisher, Cox and others at the time. Commented Jul 18 at 21:40
• Graham, I think that you are mistaken to say that Birnbaum's confidence relates to hypothesis testing because Birnbaum was intending his concept to relate to evidence. The relationship between Neyman–Pearsonian hypothesis tests and evidence is very limited indeed. Commented Jul 18 at 23:20
• @MichaelLew I enjoyed reading your paper! You successfully undermined Birnbaum’s ‘counterexample’ to the law of likelihood, and hence to his extended 1969 version of the likelihood principle which combined the standard likelihood principle with the law of likelihood (ELP = LP + LL). Commented Jul 22 at 20:36