2
$\begingroup$

This is more a conceptual question, but it seems to me that a sufficient statistic for a parameter is a concepts that applies only if we want to estimate the parameter via maximum likelihood. Is this right or wrong?

People usually claim that sufficient statistics are a data reduction technique. That is, I can store only the summary data expressed by the form of the observed value of the sufficient statistic and I will have enough information to make inference about the parameter of interest, $\theta$, in the future. To me, this is true only if we will make inference via maximum likelihood.

To expand, I might be able to come up with a better estimator for $\theta$ which takes on a strange functional form, and maybe then the observed value of the sufficient statistic will not be enough to allow me to infer about $\theta$ in the way that I wish.

$\endgroup$
5
  • 1
    $\begingroup$ We ought first to establish that this question actually has some content. Could you provide us an example of any reasonable statistical procedure that requires more information than the likelihood? $\endgroup$
    – whuber
    Jan 26, 2018 at 18:54
  • $\begingroup$ well, if I think of the method of moments estimator, there is no guarantee that the resulting method of moments estimator will be a function of the sufficient statistic, is there? $\endgroup$
    – DanRoDuq
    Jan 26, 2018 at 19:30
  • 3
    $\begingroup$ You might want to look at the Rao-Blackwell Theorem en.wikipedia.org/wiki/Rao–Blackwell_theorem for more information about the usefulness of sufficient statistics. $\endgroup$
    – jbowman
    Jan 26, 2018 at 19:33
  • 1
    $\begingroup$ Hmm interesting, so given an estimator g(x), obtained whichever way I want, then I can construct one that is a function of a sufficient statistic via E[g(x)|T(x)] and it will be no worse in terms of the expected square distance. This is nice, but now we have had to restrict what we mean be a "good estimator" which is nowhere considered when talking about sufficient statistics. $\endgroup$
    – DanRoDuq
    Jan 26, 2018 at 19:45
  • 1
    $\begingroup$ That's a fair criticism. It gets to the heart of the problem many people have with maximum likelihood in the first place: it doesn't account at all for any loss function. However, expected squared loss enjoys some rather general properties. Furthermore, anyone using MM estimators isn't considering a loss function either, so the same criticism can (and should be) be leveled at them. $\endgroup$
    – whuber
    Jan 26, 2018 at 21:23

1 Answer 1

0
$\begingroup$

It is not true that sufficient statistics are only relevant with maximum likelihood estimators. For instance, knowing a sufficient statistic is enough for Bayesian inference, see the calculation in Reducing dimension $P(\theta|y) = P(\theta|s)$ in the posterior distribution.

People usually claim that sufficient statistics are a data reduction technique. That is, I can store only the summary data expressed by the form of the observed value of the sufficient statistic and I will have enough information to make inference about the parameter of interest, 𝜃, in the future. To me, this is true only if we will make inference via maximum likelihood.

See the Rao-Blackwell theorem: Understanding the Rao-Blackwell Theorem and at Wikipedia.

Some other relevant posts:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.