2
$\begingroup$

Is there a relationship between the applicability of Maximum Likelihood Estimation and Pitman-Koopman-Darmois Theorem?

I mean if the dimensionality of the sufficient statistics depend on the sample size, does this mean MLE cannot be applied? Clearly MLE can be applied to say estimating parameters of Student-t distribution, which is not from exponential family.

So what is the connection, if there is any?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
+50
$\begingroup$

No, there's not.

Consider the negative binomial distribution with both parameters unknown. Its (minimal but not complete) sufficient statistic is the (vector of) order statistics (http://gdz.sub.uni-goettingen.de/dms/load/img/?PID=GDZPPN002461064&physid=PHYS_0353 ; note, you have to go to the page after the book review to see the paper with the proof.) Evidently the dimensionality of this statistic depends on the sample size and is unbounded as the sample size goes to infinity. Yet a maximum likelihood estimator exists.

When we actually use an estimator / calculate an estimate, we are dealing with a fixed sample. At that point, the dimensionality of the sufficient statistic is fixed as well. It's certainly nicer when we have a sufficient statistic of fixed dimension as the sample size increases, but by no means essential.

Improve this answer
$\endgroup$
10
  • $\begingroup$ I always thought that estimators for the parameters of a distribution qualified as sufficient statistics. Negative binomial has two parameters. Does this mean if I have an estimator for each parameter, they will not be jointly sufficient? $\endgroup$
    – Cagdas Ozgenc
    Nov 29, 2017 at 7:32
  • 2
    $\begingroup$ Nope! In fact, Pitman-Koopman-Darmois provides the basis for a counterexample! P-K-D says that only for exponential family distributions is the minimal sufficient statistic of bounded dimension as the sample size increases, but the Negative Binomial is not a member of the exponential family. Therefore, the sufficient statistic is not of bounded dimension as the sample size increases. However, since the N.B. only has two parameters regardless of sample size, and "two" is definitely bounded, it cannot be that the parameter estimates are jointly sufficient. $\endgroup$
    – jbowman
    Nov 29, 2017 at 17:06
  • 1
    $\begingroup$ There's nothing wrong with using the MLE for a non-exponential family distribution, it's not clear to me why you would think so. Just because the distribution is not a member of the exponential family doesn't mean the MLE doesn't have all the nice asymptotic properties. It only means you won't be able to calculate the MLE using only a small (bounded) number of sufficient statistics. Note that in order to calculate those sufficient statistics, you'll still need the whole sample, so it's not like a bounded number of sufficient statistics is any great guarantor of computational efficiency. $\endgroup$
    – jbowman
    Nov 29, 2017 at 18:50
  • 1
    $\begingroup$ I didn't mean to imply that you wouldn't be able to calculate the MLE, I meant that you wouldn't be able to calculate it with a bounded number of sufficient statistics. I should have emphasized that this is because there isn't a bounded number of sufficient statistics (given the assumption that the dist'n isn't a member of the exponential family), not because you can't do the calculation with however many sufficient statistics you have (in some cases equal to the sample size). Sorry for the lack of clarity! $\endgroup$
    – jbowman
    Nov 29, 2017 at 21:58
  • 1
    $\begingroup$ (+1) NB There's a qualification in the PKD theorem - it applies only for families of distributions where the parameter doesn't determine the support. So the sample maximum is sufficient for $\theta$ in the family of uniform distributions $\mathcal{U}(0,\theta)$. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Nov 30, 2017 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.