3
$\begingroup$

I am modeling ridership data for specific routes by month over a number of years. Some routes have as little as about 1000 riders per month while other routes may have over 20,000 riders per month. I have been looking at different approaches to model this data including a panel data model and a generalized linear data model (poisson family). However, I have found some information that says you should only use the poisson family when you have a small range in data for the y variable.

Is there a better approach to modeling count data with a large range of y values than Poisson?

$\endgroup$
1
3
$\begingroup$

For count data it is indicated (for reasons of interpretability of estimated parameters) to use a generalized linear model (GLM) with logarithmic link function, see my answer to Goodness of fit and which model to choose linear regression or Poisson

But the distribution family can be choosen in different ways. The reason for the advice you refer, to use Poisson regression when the counts are not to large, is that usually with large counts there is overdispersion, that is, the variance is larger than the variance of the Poisson.

That can be solved in various ways, like using (in R terminology) a quasipoisson family, which can be good enough. Or you can use a negative binomial family. Or, especially if you only wants predictions from the model and not need interpretable parameters, use the traditional way of a usual linear model (that is, identity link function) for the response $\sqrt{Y}$. The square-root transformation is (approximately) variance stabilizing for the Poisson (and quasi-Poisson) families. Look at Why is the square root transformation recommended for count data? for an explanation of this!

For more about overdispersion, see Modelling a Poisson distribution with overdispersion and Comparing overdispersion distributions

$\endgroup$
2
  • $\begingroup$ Thanks for you input. Just to clarify, When you mean using the square root, do you mean transforming the y by taking it sqrt and then running the model correct? Running the model with an in transformed y did show extreme over dispersion, but running it with a neg binomial got it just under 1 but real close. $\endgroup$ – CooperBuckeye05 Aug 15 '15 at 1:39
  • $\begingroup$ yes, I mean using as the response in a usual (least squares) linear model $\sqrt{Y}$ $\endgroup$ – kjetil b halvorsen Aug 15 '15 at 12:55
1
$\begingroup$

I am a little bit confused by the question (and the answer) as Box and Jenkins introduced ARIMA modelling by analyzing count data i.e. the number of people that flew in a particular month with their airline series. If the time series has small count numbers e.g. 0.0,1,1,0,1,0,2,1,0,0,1,2,1,2,0,1,..... there are definite problems/limitations applying ARIMA modelling procedures. I have been reasonably happy with count data that can average 5 or more as the assumption of continuity is less strained. Truncating/rounding the forecast is a way of providing an answer that must be an integer as integers only exist in the history.

$\endgroup$
8
  • $\begingroup$ The airline data did'nt have covariables (?), yes? So the interpretation problems do not arise. Still, a square root transformation to stabilize variance could be useful, maybe. But, of course Box knew very well about variance stabilizing transformations, if he didnt use one he must have had reasons. $\endgroup$ – kjetil b halvorsen Aug 15 '15 at 14:20
  • 1
    $\begingroup$ See my most favored/loved response as to when and why stats.stackexchange.com/questions/18844/… . He inadvertently thought that a log transformation was appropriate/needed where autobox.com/cms/index.php/afs-university/intro-to-forecasting/… suggests that a few outliers were simkply needed . Th issue of co-variables is not relevant in my oipnion. $\endgroup$ – IrishStat Aug 15 '15 at 14:54
  • $\begingroup$ My idea with mentioning covariables is that without covariables, the parmeters just represent means, or differences of means. Hardly any interpretation problems. With covariables the interpretation problem with transformation grows. $\endgroup$ – kjetil b halvorsen Aug 18 '15 at 8:40
  • 1
    $\begingroup$ You are correct in that without covariables we are just using the history of the observed series which may entail means/differences/lagged values/level shifts/time trends/seasonal pulses etc. . With covariables we can possibly incorporate supporting/helping/user-specified predictors. In both cases variance stabilizing transformations be they power transforms/weighted least squares/garch transforms may be needed to generate model errors that have constant variance and a zero constant mean. $\endgroup$ – IrishStat Aug 18 '15 at 10:10
  • $\begingroup$ Yes. My point is/was is that with covariables, we might be better off transforming fitted values than observations. In practice that is done via a GLM/link function. $\endgroup$ – kjetil b halvorsen Aug 18 '15 at 12:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.