Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I understand that for certain datasets such as voting it performs better. Why is Poisson regression used over ordinary linear regression or logistic regression? What is the mathematical motivation for it?

share|improve this question

5 Answers 5

up vote 23 down vote accepted

Poisson distributed data is intrinsically integer-valued, which makes sense for count data. Ordinary Least Squares (OLS, which you call "linear regression") assumes that true values are normally distributed around the expected value and can take any real value, positive or negative, integer or fractional, whatever. Finally, logistic regression only works for data that is 0-1-valued (TRUE-FALSE-valued), like "has a disease" versus "doesn't have the disease". Thus, the Poisson distribution makes the most sense for count data.

That said, a normal distribution is often a rather good approximation to a Poisson one for data with a mean above 30 or so. And in a regression framework, where you have predictors influencing the count, an OLS with its normal distribution may be easier to fit and would actually be more general, since the Poisson distribution and regression assume that the mean and the variance are equal, while OLS can deal with unequal means and variances - for a count data model with different means and variances, one could use a negative binomial distribution, for instance.

share|improve this answer
5  
Note that just fitting using OlS doesn't require normality - it's when you do inference on the parameters that you need the normal distribution asssumption –  Dason Mar 22 '12 at 16:20
    
@Dason: I stand corrected. –  Stephan Kolassa Jul 27 '12 at 20:45
    
If you use the Huber/White/Sandwich estimator of variance, you can relax the mean-variance assumption –  Dimitriy V. Masterov Oct 4 '13 at 2:25
    
@Dason While it's not strictly required, using the right form of model for what you're fitting almost always gives a better estimate, and you can see it in plots of residuals. –  Joe Jun 30 at 20:53

Essentially, it's because linear and logistic regression make the wrong kinds of assumptions about what count outcomes look like. Imagine your model as a very stupid robot that will relentlessly follow your orders, no matter how nonsensical those orders are; it completely lacks the ability to evaluate what you tell it. If you tell your robot that something like votes is distributed continuously from negative infinity to infinity, that's what it believes votes are like, and it might give you nonsensical predictions (Ross Perot will receive -10.469 votes in the upcoming election).

Conversely, the Poisson distribution is discrete and positive (or zero... zero counts as positive, yes?). At a very minimum, this will force your robot to give you answers that could actually happen in real life. They may or may not be good answers, but they will at least be drawn from the possible set of "number of votes cast".

Of course, the Poisson has its own problems: it assumes that the mean of the vote count variable will also be the same as its variance. I don't know if I've ever actually seen a non-contrived example where this was true. Fortunately, bright people have come up with other distributions that are also positive and discrete, but that add parameters to allow the variance to, er, vary (e.g., negative binomial regression).

share|improve this answer

Mathematically if you start with the simple assumption that the probability of an event occurring in a defined interval $T = 1$ is $\lambda$ you can show the expected number of events in the interval $T = t$ is is $\lambda.t$, the variance is also $\lambda.t$ and the probability distribution is
$$p(N=n) = \frac{(\lambda.t)^{n}e^{-\lambda.t}}{n!}$$

Via this and the maximum likelihood method & generalised linear models (or some other method) you arrive at Poisson regression.

In simple terms Poisson Regression is the model that fits the assumptions of the underlying random process generating a small number of events at a rate (i.e. number per unit time) determined by other variables in the model.

share|improve this answer

My understanding is primarily because counts are always positive and discrete, the Poisson can summarize such data with one parameter. The main catch being that the variance equals the mean.

share|improve this answer

Others have basically said the same thing I'm going to but I thought I'd add my take on it. It depends on what you're doing exactly but a lot of times we like to conceptualize the problem/data at hand. This is a slightly different approach compared to just building a model that predicts pretty well. If we are trying to conceptualize what's going on it makes sense to model count data using a non-negative distribution that only puts mass at integer values. We also have many results that essentially boil down to saying that under certain conditions count data really is distributed as a poisson. So if our goal is to conceptualize the problem it really makes sense to use a poisson as the response variable. Others have pointed out other reasons why it's a good idea but if you're really trying to conceptualize the problem and really understand how data that you see could be generated then using a poisson regression makes a lot of sense in some situations.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.