Why Specifically Use Poisson Regression For Count Data?

Question

I am a MBA Student that is taking some statistics courses, a colleague recommended this site as a useful resource! So far there seems to be a lot of interesting information here! I posted a previous question on the same topic and was advised to break it into individual parts.

We are learning about "Count Data" - for example, we are interested in learning how to make a model that predicts the number of complaints a customer might file. In a previous stats course, I learned about basic regression models and the Poisson Distribution. Now, I am trying to understand how these two can be put together!

• Why should Poisson Regression be used for Count Data instead of a "vanilla linear regression"? I understand the basic argument : Count Data is by definition discrete and you would rather use a model in which predictions are always discrete (i.e. Poisson Regression) ... but to me, this seems like a formality. Couldn't I just use a linear regression model and round the predictions to the nearest whole number?

Answer 1

Because you're an MBA student and not a statistician, I will list a couple of reasons which may be applicable.

• First, note that expected number of complaints can not be negative. A customer can not complain -2 times on average. So there is an explicit requirement that the prediction be positive. It need not necessarily be an integer (because regression predicts the expected number of complaints, and expectations or averages need not be integers). The poisson makes this natural since the canonical link is the natural log, meaning $E[y] = \exp(X\beta)$.

• You may say, "well, I can just take the log of my counts and presto, I can use OLS". This could work, say if there are a moderate number of complaints. However if you include customers who issue 0 complaints, then we have a problem since $\log(0)$ is undefined. Again, this is no problem for the poisson. The support of the poisson distribution is on the non-negative integers, so no matter what $\lambda = \exp(X\beta)$ is, 0 is always a possibility though it may not be probable. Excluding the 0s would mean you're modelling the expected number of complaints for those people who complain, which is different. You could do the $\log(1+y)$ trick, but its just more details to keep track of unnecessarily in my opinion.

• Lastly, were you to adopt a Bayesian perspective and compute a prediction interval, the prediction interval will necessarily be within a realistic range without you having to intentionally force it to be non-negative.

Answer 2

Short version:

Poisson regression deals with some of the quirks of modeling count rates, including the requirement that rates be positive and the presence of zero count cases in the data, which will cause problems in ordinary least squares.

Longer version:

When you are dealing with count data, you will generally assume that the counts come in at some average rate (which is possibly affected by one or more independent variables -- a.k.a. "covariates"), but the actual counts that you see will vary around that average. This variation is assumed to be truly random, not predictable even in principle, so your model is going to try to predict that average, and in particular it's going to try to tease out the influence of your covariates on the average. With that in mind, let's think about how we might model the system.

The first thing we'll note is that although the counts are integers, the average need not be. It's perfectly fine to have an average rate of 3.5 counts per minute, even though we will observe an integer number of counts in each actual minute, so the discreteness isn't going to cause us any problems, which is good.

The second thing we have to consider is that the average rate must be positive, which is problematic for ordinary least squares (OLS). If the counts are low, and the effect of the covariates is strong, then we could easily end up with a model that predicts a negative or zero rate, which is not good. One way around this problem is, instead of having our model predict the average count rate, we could have it predict the log of the rate. The log of the rate can be any real number, so we don't have to worry about our model giving impossible results. So, you could do an OLS fit to the log of the counts and interpret the model predictions as the log of the rate.

Unfortunately, there's still a problem. If the rate is low, then you will occasionally get zero counts, and with the log of zero being $-\infty$, these cases are going to mess up your fit. You could try setting a small positive number, say $10^{-8}$ as a minimum, but if you do that then you'll find that your answer depends on the precise value that you select for that cutoff. Here's where Poisson regression comes to the rescue.

In nature, a lot of counting processes show a specific relationship between the average number of counts and the distribution of observed counts. That relationship is called the Poisson distribution, and if we call the average rate $\lambda$, then the Poisson distribution looks like this: $$ P(y=k; \lambda) = \frac{\lambda^k e^{-\lambda}}{k!}. $$ The important thing here is that P(y=0) doesn't blow up this expression; it gives a perfectly well-behaved number.

So, the name of the game is that we let $\lambda$ be a function of our coefficients: $$ \log(\lambda) = \alpha + \beta_1 x_1 + \beta_2 x_2 + \ldots, $$ and we solve for coefficients $\alpha$, $\beta_1$, etc. that maximize $P(y_i = k_i; \lambda_i)$ for all of our observations. This takes care of the requirement that $\lambda > 0$, deals with any zero-count observations, and as an added bonus accounts for the fact that for low rates the distribution of observed counts tends to be rather skewed (OLS is based on the normal distribution, which is symmetric). All of these are desirable, and that is why Poisson regression is preferred to OLS for count data.

These kinds of models, by the way, are called Generalized Linear Models (GLMs), and Poisson regression is just one possibility. For example, if we had started with the assumption that our observations were True/False observations of a binary experiment, and that our model had to produce probabilities bounded by 0 and 1, we would have arrived at a formula for logistic regression, a type of GLM for predicting probabilities of binary experiments. There are a whole slew of GLM formulations out there for different kinds of problems, with Poisson regression and logistic regression being the most common.