You can use maximum likelihood estimation to estimate the regression parameters for a random variable with Poisson or Binomial distributions, but I haven't heard of a chi-squared regression or a Gamma function regression. What makes the Poisson and Binomial special enough to have their own regression techniques?
3
-
5$\begingroup$ There's also beta regression, negative binomial regression, and others I've forgotten. You don't tend to get chi-square distributed data. $\endgroup$– Jeremy MilesCommented Apr 30 at 0:39
-
7$\begingroup$ One might say that distributions like the negative binomial distribution and Poisson distribution really count. 🥁 $\endgroup$– GalenCommented Apr 30 at 1:16
-
4$\begingroup$ There's certainly gamma regression (and indeed chi-squared, as a special case by specifying the dispersion parameter, though it typically won't have a lot of applications), since gamma is one of the standard distributions available for generalized linear models. I've done gamma regression many times (many - there's lots of situations where it's a much better model than assuming conditional normality). Just because you haven't heard of a thing doesn't mean it doesn't exist. I also use many of the lifetime distributions available for survival analysis in models that are not about survival times $\endgroup$– Glen_bCommented Apr 30 at 3:03
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Poisson and Binomial are reasonable distributions for raw data values for a response variable: Y = integer counts and Y = binary yes/no answers are fairly common. There are also Beta regression (Y = proportions between 0 and 1), as well as Exponential, Gamma, and Weibull regressions (Y = waiting times or survival times, though a course on survival analysis might focus on other methods first). As @JeremyMiles mentions in a comment, there's also Negative Binomial regression (Y = number of failures until a certain number of successes).
By contrast, some commonly-taught distributions like chi-squared or F are designed to be good models for the distribution of statistics rather than data values. Chi-squared is a sum of squared Gaussians, and F is a ratio of scaled sums of squares. So these are useful as null distributions for variances, MSEs, and so on, but it's quite rare for Y = variances or Y = MSEs to be treated as the data values that we're trying to predict in a regression setting.
$\begingroup$
$\endgroup$
4
Another aspect of this is that the generalised linear models including linear, Poisson, and binomial, are computationally well-behaved (even fitted with Newton's method) and so were easy to implement and reliable to run in the Old Days.
Many regression models that are perfectly reasonable from a statistical viewpoint are substantially less well-behaved computationally and require a lot more care in programming.
[This was also one of the advantages of the Cox proportional hazards model in survival, which is computationally easier to handle than, say, Weibull models]
answered Apr 30 at 1:09
-
$\begingroup$ This answer could be improved be mentioning exponential families, perhaps? that is the main reason GLMs are computationally "nice". $\endgroup$– JDLCommented Apr 30 at 11:48
-
$\begingroup$ yeah, the is a lack of mentioning exponential families in all answers $\endgroup$ Commented Apr 30 at 15:08
-
2$\begingroup$ @JDL only up to a point. The Gamma family with canonical link isn't that well-behaved, nor (notoriously) is the Binomial with log link, nor are various multlivariate exponential family models. And on the other hand, the Cox model isn't an exponential family but is computationally simple and very popular. $\endgroup$ Commented Apr 30 at 20:16
-
$\begingroup$ Fair point, except that log isn't the canonical link for binomial. $\endgroup$– JDLCommented May 1 at 15:16