2
$\begingroup$

Let's assume that I have a set of predictors and a non-negative integer resulting variable (number of events). All observations are repeated few times (it means that all predictors have the same values more than once). I need to predict an average number of events for every possible combination of predictor's values. I combined all observations with the same predictors' values to one, and assigned an average number of events for all these observations to the new one.

Next, I built four different models - OLS, OLS with transformed resulting variable, hurdle Gamma GLM and, I don't know why, Poisson GLM. Surprisingly, Poisson was the best one. Since this is my final qualification thesis, I need some theoretical basis, but I can't figure one, I've been always thinking that Poisson regression assumes integer data. Hope, somebody could help.

Improve this question
$\endgroup$
4
  • $\begingroup$ When you say "best one", in what sense was it best, exactly? $\endgroup$
    – Glen_b
    Commented May 7, 2014 at 2:22
  • $\begingroup$ Prediction error $\endgroup$
    – Evgenii Nikitin
    Commented May 7, 2014 at 11:08
  • $\begingroup$ mean square prediction error? something else? $\endgroup$
    – Glen_b
    Commented May 7, 2014 at 12:40
  • 1
    $\begingroup$ MAE, RMSE, cross-validated MAE $\endgroup$
    – Evgenii Nikitin
    Commented May 7, 2014 at 18:56

2 Answers 2

Reset to default
2
$\begingroup$

Take a look at the references in this answer for why a robust poisson model can be applied to non-integer data.

You can also motive it in your case by saying you're modeling a rate per covariate duplicate, as in this question with time. On the other hand, I don't really see a need to aggregate. The Poisson model gives you the expected value conditional on covariates, so it's OK to have duplicates with different outcomes but same covariates.

Improve this answer
$\endgroup$
10
  • $\begingroup$ First of all, thanks for the references, I'll take a look at them right now. Then, about why I need to aggregate. If I try to predict every observation outcome separately, obviously the average error is higher, there are some effects for duplicates that I don't consider in my model. Since I don't need predictions for every outcome, I don't want to report higher errors. Probably, I'm wrong here, so, please, correct me in this case. $\endgroup$
    – Evgenii Nikitin
    Commented May 6, 2014 at 20:25
  • $\begingroup$ Indeed, I'm modeling a rate per duplicate, but I can't find anything about that in quoted question. Isn't it about an exposure variable? $\endgroup$
    – Evgenii Nikitin
    Commented May 6, 2014 at 20:49
  • 1
    $\begingroup$ Your outcome is the total number for a set of duplicates. The exposure variable is the number of duplicates. $\endgroup$
    – dimitriy
    Commented May 6, 2014 at 21:03
  • $\begingroup$ I am not sure I follow your logic about the duplicates. Let me think some more about this. $\endgroup$
    – dimitriy
    Commented May 6, 2014 at 21:05
  • $\begingroup$ Oh, that's a nice idea. I'll try it right now. Thanks. $\endgroup$
    – Evgenii Nikitin
    Commented May 6, 2014 at 21:06
1
$\begingroup$

It seems your resulting variable is a non-negative integer, which is the support of the Poisson distribution. So your question doesn't really match the title of the post (I'm confused).

As far as theoretical justification, mostly what you'd have to do is show that the events (which have integer counts) follow a Poisson Process, which has a few simple properties.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Well, my variable was a non-negative integer before I combined observations and substituted the number of events with an AVERAGE number of events, which is obviously mostly non-integer. I don't need to predict the number of events in the every particular case since there are some random effects that I can't consider in my model, I just need an average number for a certain set of conditions (predictors). $\endgroup$
    – Evgenii Nikitin
    Commented May 6, 2014 at 20:14
  • $\begingroup$ Sounds like your averaging is getting an estimator for the mean-parameter of a Poisson distribution. That parameter is real-valued, even though the actual realized value of its random variable is integer-valued. It's like a coin flip: the mean outcome is 0.5 (real-valued), but the only outcomes possible are {0,1}. When you average all the 0's and 1's you get an estimator for that underlying 0.5. $\endgroup$
    – TheBigAmbiguous
    Commented May 6, 2014 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.