I understand that for certain datasets such as voting it performs better. Why is Poisson regression used over ordinary linear regression or logistic regression? What is the mathematical motivation for it?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ See my answer to this post for another viewpoint: stats.stackexchange.com/questions/142338/… $\endgroup$– kjetil b halvorsen ♦Commented Apr 24, 2015 at 21:18
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
4
Poisson distributed data is intrinsically integer-valued, which makes sense for count data. Ordinary Least Squares (OLS, which you call "linear regression") assumes that true values are normally distributed around the expected value and can take any real value, positive or negative, integer or fractional, whatever. Finally, logistic regression only works for data that is 0-1-valued (TRUE-FALSE-valued), like "has a disease" versus "doesn't have the disease". Thus, the Poisson distribution makes the most sense for count data.
That said, a normal distribution is often a rather good approximation to a Poisson one for data with a mean above 30 or so. And in a regression framework, where you have predictors influencing the count, an OLS with its normal distribution may be easier to fit and would actually be more general, since the Poisson distribution and regression assume that the mean and the variance are equal, while OLS can deal with unequal means and variances - for a count data model with different means and variances, one could use a negative binomial distribution, for instance.
answered Sep 23, 2010 at 20:42
-
22$\begingroup$ Note that just fitting using OlS doesn't require normality - it's when you do inference on the parameters that you need the normal distribution asssumption $\endgroup$– DasonCommented Mar 22, 2012 at 16:20
-
2$\begingroup$ @Dason: I stand corrected. $\endgroup$ Commented Jul 27, 2012 at 20:45
-
3$\begingroup$ If you use the Huber/White/Sandwich estimator of variance, you can relax the mean-variance assumption $\endgroup$– dimitriyCommented Oct 4, 2013 at 2:25
-
1$\begingroup$ @Dason While it's not strictly required, using the right form of model for what you're fitting almost always gives a better estimate, and you can see it in plots of residuals. $\endgroup$– JoeCommented Jun 30, 2014 at 20:53
$\begingroup$
$\endgroup$
Essentially, it's because linear and logistic regression make the wrong kinds of assumptions about what count outcomes look like. Imagine your model as a very stupid robot that will relentlessly follow your orders, no matter how nonsensical those orders are; it completely lacks the ability to evaluate what you tell it. If you tell your robot that something like votes is distributed continuously from negative infinity to infinity, that's what it believes votes are like, and it might give you nonsensical predictions (Ross Perot will receive -10.469 votes in the upcoming election).
Conversely, the Poisson distribution is discrete and positive (or zero... zero counts as positive, yes?). At a very minimum, this will force your robot to give you answers that could actually happen in real life. They may or may not be good answers, but they will at least be drawn from the possible set of "number of votes cast".
Of course, the Poisson has its own problems: it assumes that the mean of the vote count variable will also be the same as its variance. I don't know if I've ever actually seen a non-contrived example where this was true. Fortunately, bright people have come up with other distributions that are also positive and discrete, but that add parameters to allow the variance to, er, vary (e.g., negative binomial regression).
answered Sep 23, 2010 at 20:52
$\begingroup$
$\endgroup$
Mathematically if you start with the simple assumption that the probability of an event occurring in a defined interval $T = 1$ is $\lambda$ you can show the expected number of events in the interval $T = t$ is is $\lambda.t$, the variance is also $\lambda.t$ and the probability distribution is
$$p(N=n) = \frac{(\lambda.t)^{n}e^{-\lambda.t}}{n!}$$
Via this and the maximum likelihood method & generalised linear models (or some other method) you arrive at Poisson regression.
In simple terms Poisson Regression is the model that fits the assumptions of the underlying random process generating a small number of events at a rate (i.e. number per unit time) determined by other variables in the model.
answered Sep 23, 2010 at 23:28
$\begingroup$
$\endgroup$
Others have basically said the same thing I'm going to but I thought I'd add my take on it. It depends on what you're doing exactly but a lot of times we like to conceptualize the problem/data at hand. This is a slightly different approach compared to just building a model that predicts pretty well. If we are trying to conceptualize what's going on it makes sense to model count data using a non-negative distribution that only puts mass at integer values. We also have many results that essentially boil down to saying that under certain conditions count data really is distributed as a poisson. So if our goal is to conceptualize the problem it really makes sense to use a poisson as the response variable. Others have pointed out other reasons why it's a good idea but if you're really trying to conceptualize the problem and really understand how data that you see could be generated then using a poisson regression makes a lot of sense in some situations.
$\begingroup$
$\endgroup$
My understanding is primarily because counts are always positive and discrete, the Poisson can summarize such data with one parameter. The main catch being that the variance equals the mean.