2
$\begingroup$

An exponential family (under natural parameterization) is such that $p(X|\eta)=h(X)\exp\{\eta^\top T(X)-A(\eta)\}$, where $X$ is the data, $\eta$ is the natural parameter, and $h,T,A$ are some functions (that are interrelated).

The statistic $T(X)$ is sufficient, and the maximum likelihood estimator is $\hat\eta(X)$ is such that $\nabla A(\hat\eta(X))=T(X)$.

It therefore seems to me that the density can be written as $p(X|\eta)=h(X)\exp\{\eta^\top\nabla A(\hat\eta(X))-A(\eta)\}=h(X)\cdot f(\hat\eta(X),\eta)$, which entails that $\hat\eta(X)$ is a sufficient statistic.

Am I mistaken or is it sound to conclude that a maximum likelihood estimator in an exponential family is always sufficient?

$\endgroup$
4
  • $\begingroup$ Thank you. Which part of the "proof" fails if the parameterization is not minimal? $\endgroup$
    – erezmb
    Commented Aug 10, 2020 at 8:43
  • $\begingroup$ Thanks! Is it possible to point me to a textbook or journal reference? $\endgroup$
    – erezmb
    Commented Aug 10, 2020 at 9:24
  • 3
    $\begingroup$ I deem your proof is enough: Provided$$\nabla A(\hat\eta(t))=t\tag{1}$$has one and only one solution$$\hat\eta(t)$$for almost every realisation $t$ of $T(X)$, the factorisation theorem applies. Cases where this condition fails may be for discrete exponential families, on the boundary of the support of $T(X)$ and curved exponential families when the constraints on $\eta$ may be incompatible with (1). (But it is debatable this is a "natural" exponential family.) $\endgroup$
    – Xi'an
    Commented Aug 10, 2020 at 9:39
  • $\begingroup$ Michael Jordan mentions the sufficiency of the mean parameter MLE in his notes.. $\endgroup$
    – Xi'an
    Commented Aug 10, 2020 at 9:40

1 Answer 1

1
$\begingroup$

You can also turn it around. If the MLE is a (one-to-one) function of the sufficient statistic $\sum T(X_i)$, then it is itself a sufficient statistic as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.