# Sufficient Statistic and Maximum likelihood

This is more a conceptual question, but it seems to me that a sufficient statistic for a parameter is a concepts that applies only if we want to estimate the parameter via maximum likelihood. Is this right or wrong?

People usually claim that sufficient statistics are a data reduction technique. That is, I can store only the summary data expressed by the form of the observed value of the sufficient statistic and I will have enough information to make inference about the parameter of interest, $\theta$, in the future. To me, this is true only if we will make inference via maximum likelihood.

To expand, I might be able to come up with a better estimator for $\theta$ which takes on a strange functional form, and maybe then the observed value of the sufficient statistic will not be enough to allow me to infer about $\theta$ in the way that I wish.

• We ought first to establish that this question actually has some content. Could you provide us an example of any reasonable statistical procedure that requires more information than the likelihood?
– whuber
Jan 26, 2018 at 18:54
• well, if I think of the method of moments estimator, there is no guarantee that the resulting method of moments estimator will be a function of the sufficient statistic, is there? Jan 26, 2018 at 19:30
• You might want to look at the Rao-Blackwell Theorem en.wikipedia.org/wiki/Rao–Blackwell_theorem for more information about the usefulness of sufficient statistics. Jan 26, 2018 at 19:33
• Hmm interesting, so given an estimator g(x), obtained whichever way I want, then I can construct one that is a function of a sufficient statistic via E[g(x)|T(x)] and it will be no worse in terms of the expected square distance. This is nice, but now we have had to restrict what we mean be a "good estimator" which is nowhere considered when talking about sufficient statistics. Jan 26, 2018 at 19:45
• That's a fair criticism. It gets to the heart of the problem many people have with maximum likelihood in the first place: it doesn't account at all for any loss function. However, expected squared loss enjoys some rather general properties. Furthermore, anyone using MM estimators isn't considering a loss function either, so the same criticism can (and should be) be leveled at them.
– whuber
Jan 26, 2018 at 21:23

It is not true that sufficient statistics are only relevant with maximum likelihood estimators. For instance, knowing a sufficient statistic is enough for Bayesian inference, see the calculation in Reducing dimension $P(\theta|y) = P(\theta|s)$ in the posterior distribution.