3
$\begingroup$

Sufficiency Principle as defined in Casella:

enter image description here

Where Sufficient Statistic is defined as:

enter image description here

Question: Is the Sufficiency Principle an axiom?

My thoughts and research so far:

  1. I'm uncertain if the Sufficiency Principle is even a mathematical statement, because its not clear if "inference" is a mathematical object.
  2. According to this answer on a question about "Weak Likelihood Princple" (which I believe is an alias for Sufficiency Principle), it's stated that W.L.P. is "axiomatic" but not a "mathematical claim".

Birnbaum (1962) kicked off the approach of giving a formal account of the relationships between various "principles". He took the concept of evidential meaning as basic & the W.L.P. as axiomatic, & went on to derive the Strong Likelihood Principle from this & another axiom, the Conditionality Principle. His formal statement of the W.L.P. is that for inference about a parameter $\theta$ in an experiment $E$, where $T$ is a sufficient statistic for $\theta$, if $T(x) = T(y)$ for samples $x$ & $y$, then $\operatorname{Ev(E,x)} = \operatorname{Ev}(E, y)$; in which $\operatorname{Ev(E,x)} = \operatorname{Ev}(E, y)$ denotes "evidential equivalence" or your "containing the same inferentially useful information". This is not an empirical claim, or even a mathematical one, but purports to constrain (sensible) inferential procedures: if it's entailed by other foundational principles you hold dear, then all well & good; if not then you may try & balance it against those or to eschew it altogether.

  1. The definition of sufficient statistic above is specifically for classical or F-Sufficiency. Alternatively we can define a Bayes or B-Sufficient statistic (wiki) as:

enter image description here

  1. Using this alternative definition of sufficient statistic, and defining "inference" as posteriori probability of $\theta$, ie inference = $\Pr(\theta\mid X=x)$, the sufficiency principle is both mathematical and can be proved trivially by definition of sufficient statistic.

Which brings us to, Question 2: Is the answer to Question 1 different for a Bayesian vs Frequentist?

$\endgroup$

1 Answer 1

6
$\begingroup$

I think the term "Principle" works well for the Sufficiency Principle (but see comments;-). One can use it as a guidance for constructing methods of inference in many situations, and some mathematical theory (like the Rao-Blackwell Theorem) in fact shows that in order to construct methods of inference that are in a certain sense optimal under the model, the Sufficiency Principle has to be respected.

That said, I wouldn't call it "Axiom" in the sense that it doesn't express something that "is generally taken to be true" or is a "starting point" for a formal system (cp. Wikipedia on Axioms). The Sufficiently Principle in my view has authority only to the extent to which it actually is backed up by proper mathematical theory, see above.

In fact the Sufficiency Principle relies very strongly on the truth of the assumed model, which in reality should never be taken for granted, and it is known in Robust Statistics that if you want to have inference that is still good under the assumed model but doesn't break down in certain neighbourhoods of it, you need to violate the Sufficiency Principle in many cases.

So the Sufficiency Principle can be argued if you want your inference method to have certain optimality properties assuming the model, but if you are interested in other practically relevant criteria such as robustness, you better be prepared to drop it. (It's similar with the WLP - as long as it gives you all you want, that's fine, but if you discover that you want something that you can't get as long as you respect the WLP, but you can get it in other ways, the WLP has no authority to stop you from doing something else - as long as you understand the price to pay that is.)

The answer is different for Bayesian statistics to the extent that the Sufficiency Principle (and/or the WLP) is built directly into the Bayesian formalism (in which case you can't even violate it as long as you're doing Bayesian stats; but of course also here you may want to deviate from it at the point where you start questioning your models - you may have to violate coherence and therefore the essence of Bayesian reasoning to do that though).

$\endgroup$
13
  • 3
    $\begingroup$ @Shreyans Ultimately I'm not a linguist so maybe I was wrong redarding the difference of word meaning between principle and axiom. I rather had a Groucho Marx type of principle in mind, as in "These are my principles, and if you don't like them... well, I have others." ;-) $\endgroup$ Commented May 26 at 14:10
  • 1
    $\begingroup$ @Shreyans Re answer to Graham Bornholt - I'm not sure anyone uses the Sufficiency Principle as an Axiom. I think it is used mostly in the cases in which there is a sound mathematical basis for it, and then the basis is the existing mathematical justification rather than the principle itself. In that sense the principle is rather a summary of some mathematical results than the basis of anything (if maybe the basis/inspiration of certain ideas to be properly justified by maths later). $\endgroup$ Commented May 26 at 14:15
  • 1
    $\begingroup$ @ChristianHennig Thanks for the clarification, I understand the answer better now. In conclusion SP is not an axiom, but more like a guiding principle that "has authority only to the extent to which it actually is backed up by proper mathematical theory" $\endgroup$
    – Shreyans
    Commented May 26 at 21:10
  • 2
    $\begingroup$ @Shreyans We have reached a consensus. $\endgroup$ Commented May 26 at 21:35
  • 1
    $\begingroup$ @Shreyans: The S.P. is a 'starting point' for the (rudimentary) formal system constructed by Birnbaum in the referenced paper; which is the only reason I called it 'axiomatic' in my answer. $\endgroup$ Commented Jun 4 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.