5
$\begingroup$

I have a question concerning the comparison of negative binomial regression models in SPSS. I need to compare two models to check whether the second model significantly increases the fit of the model/explained variance in the dependent. The second model contains 1 extra predictor and I want to know whether this predictor significantly improves the model over the first model.

I have already tried comparing fit indices (I find only very small differences) and calculating pseudo R squares (okay, but do not indicate whether the second model is significantly better).

Is it possible to compare two negative binomial regression models aside from the fit indices? Any help?

$\endgroup$

1 Answer 1

6
$\begingroup$

You can do a likelihood ratio test. Take the difference of -2Log(Likelihood) and compare to a chi-squared distribution with 1 degree of freedom (1 df because the models differ by 1 parameter). SPSS supplies -2log(likelihood). It calls it the Deviance, and it's in an output box labelled "Goodness of Fit".

Note that this is an asymptotic test, so the accuracy of the p-value that you get from this will depend on sample size, and how well your data actually follow a negative binomial distribution.

This same approach can be used for any nested, generalized linear models. (nested means that all the parameters of the smaller model are found in the larger model).

  1. So, to recap, fit model 1 and locate the Deviance $D_1$
  2. Fit model2 and locate its Deviance, $D_2$
  3. Calculate $D_2-D_1=S$
  4. If S is larger than 3.84, reject the null hypothesis that the extra predictor is useless at the 5% level.
  5. For a p-value, use S and a set of chi-squared tables.

This gives you an hypothesis test to answer your question. Another approach would be to assess how well the respective models do in predicting outcomes. If you have a reserve set of data, you can see if the larger model does a better job of predicting outcomes. By "Better", I mean that the larger model has a meaningful impact on whatever it is you are using the model for. Strictly speaking, neither the fit indices nor a significance test actually tell you if the model fits. They just let you compare between models.

$\endgroup$
5
  • $\begingroup$ (+1) Information criteria can also be used to compare models - you've got a proper likelihood. $\endgroup$
    – Scortchi
    Commented Jun 5, 2014 at 12:44
  • 1
    $\begingroup$ @Scortchi Would you like to elaborate that for a second answer to the question? $\endgroup$
    – Placidia
    Commented Jun 5, 2014 at 12:47
  • $\begingroup$ Okay, I have calculated the difference in deviance. It's 33.03. We do have a very large sample (N = 3372). Is is than correct to conclude that the larger model is significantly "better" than the smaller one. $\endgroup$
    – nerdalert
    Commented Jun 5, 2014 at 13:44
  • $\begingroup$ @nerdalert now we're into the correct interpretation of an hypothesis test, and that can be controversial. The differences between the two models cannot be attributed to chance. Chances are, you need that additional predictor. But to see how much it matters, you would need to look at how well the two models predict the data you see. $\endgroup$
    – Placidia
    Commented Jun 5, 2014 at 14:46
  • $\begingroup$ @ Placidia : do you mean the fit of the models? $\endgroup$
    – nerdalert
    Commented Jun 6, 2014 at 8:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.