1
$\begingroup$

I have run a classical Poisson regression model and a Poisson random effect model in R. I want to know if there is any goodness of fit for the Poisson random effect model?

$\endgroup$
1
$\begingroup$

There are certainly measures of goodness of fit: You could, for example, if $P_i$ is the predicted value for each case and $Y_i$ is the actual value, you could take $\sum{P_i-Y_i}$ and scale it by some suitable measure.

Another idea is to look at a plot of the predicted values vs. actual values: Various plots could be useful, you might have to jitter $Y_i$ since it will be all integers.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Peter Flom, why not $\sum({P_i-Y_i})^2$? And why not use RMSE? Why not residual deviance reported by the glm function, and why not some pseudo R-squared formula? $\endgroup$ – Curious Mar 24 '14 at 15:33
  • 1
    $\begingroup$ The problem is that almost all such measures for GLMs are ad-hoc. In my experience, the best goodness of fit measure for Poisson distributed data is $\frac{O-E}{E}$ which has a known distribution for Poisson distributed data (and also can be interpreted as % error). $\endgroup$ – AdamO Mar 24 '14 at 15:49
1
$\begingroup$

When I think of GoF, I think of prediction. If we were interested in just inference, then we would just favor choosing more robust Poisson models, like Quasipoisson or GEE (which estimates population averaged parameters and is consistent in the presence of model misspecification). If you're estimating an RR, just use robust standard errors and be done with it.

Predicted values from mixed models have hairy interpretations. The problem is that there is no generalizability. Certainly, in the same from which you generated the data, the mixed model can interpolate data within clusters. However, random intercepts cannot be randomly estimated in new clusters.

Furthermore, the effective degrees of freedom in the mixed model versus the "fixed effects" model is difficult to reconcile. They are certainly nested, but you're estimating something extra in the mixed model. As such, any performance measure based on predicted values from the mixed model will almost certainly do better. You can think of this as the exact same thing as overfitting (although the random effect still allows for consistent estimation of the individual level RR, so no harm done there).

So, expect the mixed model to look better, but when you recognize performance based GoF measures are based on prediction, realize that the mixed model will be unable to estimate random intercepts in new data, so there is no generalizability.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.