8
$\begingroup$

In Generalized Linear Models the conditional distribution of the response variable has to belong to the exponential family. Why is this restriction important? What property would a regression model lose if we chose a distribution outside the exponential family?

$\endgroup$
2
  • 3
    $\begingroup$ sufficiency of the maximum likelihood estimate is one one thing you will lose. $\endgroup$
    – utobi
    Nov 19, 2016 at 14:30
  • 1
    $\begingroup$ Theory and algorithms are easier ti develop inside the exponential family. But you can go outside it, and it is done nowadays---see the R VGAM package, doing just this. $\endgroup$ Nov 19, 2016 at 14:36

1 Answer 1

2
$\begingroup$

Jaynes makes the argument that when you leave the exponential family, your estimators cease to be sufficient statistics. If a statistic is sufficient for a parameter then $\Pr(t|\theta)=\Pr(X|\theta)$. Implying that the information in $t$ is the same as in the sample $X$. Bayesian methods always use all the information in $X$. Non-Bayesian methods use a statistic. If that statistic contains the same information then the estimator will be no worse.

If the statistic is not sufficient, then it will be noisier than the Bayesian estimate. Bayesian estimators are always admissible statistics. If the distribution is not in the exponential family, then the Bayesian estimator will stochastically dominate it, hence the estimator will be inadmissible.

So if you do not make such an assumption, then you would be better off using a Bayesian model in all circumstances. If that were the case, why would you use an alternative?

$\endgroup$
1
  • $\begingroup$ @ Dave Harris: great answer! I had some questions about your answer, I posted them over here: stats.stackexchange.com/questions/552707/… ...maybe you can take a look at it later if you have time? thank you! $\endgroup$
    – stats_noob
    Nov 18, 2021 at 8:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.