This question came up in class: If we use p-values to evaluate hypotheses on an experiment, which part of the Likelihood Principle are we not obeying: Sufficiency or Conditionality?

My intuition would be to say Sufficiency, since computing a p-value relies on unobserved outcomes of an experiment, and Sufficiency seems to deal more with observations within a single experiment while Conditionality seems to deal more with different experiments.


Your intuition seems reasonable here, but it is worth stating things more precisely. So long as the p-value for the test is not a function of a sufficient statistic, the sufficiency principle is breached. For the conditionality principle, things are a bit trickier. The conditionality principle was originally described by Birnbaum as follows:

...the evidential meaning of any outcome of any experiment is the same as that of the corresponding outcome of the corresponding component experiment, ignoring the overall structure of the mixture experiment. [This] may be described informally as asserting the "irrelevance of (component) experiments not actually performed." (Birnbaum 1962, p. 271).

Whether or not this principle is breached by a classical hypothesis test (using a p-value as its evidentiary measure) really depends on whether or not the experiment leading to the test can be framed as a mixture of smaller component experiments, which occur conditionally on some initial results. If the experiment can be framed in this way, the p-value will depend on component experiments not performed, through the fact that it will include the probability of outcomes in these experiments that are at least as conducive to the alternative as the actual outcome of the experimental component that was performed. This would be a breach of the conditionality principle, often in addition to breaching the sufficiency condition.

| cite | improve this answer | |
  • $\begingroup$ But will there always exist a (unique) decomposition in component experiments? $\endgroup$ – kjetil b halvorsen Apr 10 '19 at 21:48
  • $\begingroup$ I presume that would depend on the experiment. $\endgroup$ – Ben Apr 10 '19 at 22:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.