Is there a difference between Bayesian and Classical sufficiency?

Question

The title pretty much says it all. I wonder whether there is any difference in the way Bayesians understand sufficiency vs. the way orthodox statistics understands sufficiency, or are they equivalent? If there is a difference, what is it?

Answer 1

In addition to the other excellent answer: The question of equivalence between Bayesian (B-) sufficiency and Classical (F-sufficiency) is answered in the abstract by NOT. But, this is based on abstract measure-theoretic definitions, and proofs are technical using measure theory. But, counterexamples to equivalence are artificial, models constructed to be counterexamples, not to be useful. As long as your sample and parameter space are (measurable) subsets of some finite-dim euclidean space, there are no counterexamples.

In short: F-sufficiency implies B-sufficiency, but not the other way. A Bayes but Not Classically Sufficient Statistic by D. Blackwell and R. V. Ramamoorthi gives the artificial example. They refer to a proof of equivalence in the dominated case, all the probability measures in the hypothesis space are dominated by some common, sigma-finite measure. That covers most models. The proof, says this authors, "follows easily from the results of Halmos and Savage".

Another interesting paper, K. K.Roy and R. V. Ramamoorthi: Relationship between Bayes, Classical and Decision Theoretic Sufficiency is working towards finding conditions under witch B-sufficiency will imply F-sufficiency. The conditions are technical, measure-theoretic conditions on the sample and parameter space and their sigma-algebras (Proposed by D Blackwell). That paper studies a special case of that condition (spaces standard Borel, sigma-algebras countably generated. "Interest in countably generated sigma-fields stems from the fact that these are precisely the sigma-fields generated by Borel measurable real valued statistics."

--

References:

[1]: A Bayes but not Classically Sufficient Statistic, The annals of statistics, Vol 10, No 3 (sept, 1982) (in JSTOR and project Euclid)

[2]: Halmos, P. R. and Savage, L. J. (1949) Applications of the Radon-Nikodym theorem to the theory of sufficient statistics. Ann. Math. Statistics 20 225--241

[3]: Sankhya, the Indian Journal of Statistics, Series A (1961--2002), 1979 Indian Statistical Institute

Answer 2

Here is one example of differentiation between classical and Bayesian statistics: when comparing two models $\mathcal{M}_1$ and $\mathcal{M}_2$, a statistic $S(\cdot)$ may be sufficient for both models, hence sufficient in a classical sense, but insufficient for model comparison as e.g. in Bayes factors, when the conditional distribution of the data given $S$ varies between models. The difference is due to the fact that the model index is a parameter from a Bayesian perspective but not a parameter from a classical one. (This is discussed further in our ABC model choice papers.)