8
$\begingroup$

I am trying to come up with a model by using negative binomial regression (negative binomial GLM). I have a relatively small sample size (greater than 300), and the data are not scaled. I noticed that there are two ways to measure goodness of fit - one is deviance and the other is the Pearson statistic. How can I determine which goodness-of-fit measure to use? Are there some criteria that I can take a look at in selecting the goodness-of-fit measure?

$\endgroup$
  • $\begingroup$ I'm not sure what you mean by "I have a relatively small sample size (greater than 300)". $\endgroup$ – Dason Nov 24 '13 at 20:21
  • $\begingroup$ @Dason 300 is not a very large number in like gene expression $\endgroup$ – SmallChess Nov 3 '15 at 6:14
10
$\begingroup$

The goodness-of-fit test based on deviance is a likelihood-ratio test between the fitted model & the saturated one (one in which each observation gets its own parameter). Pearson's test is a score test; the expected value of the score (the first derivative of the log-likelihood function) is zero if the fitted model is correct, & you're taking a greater difference from zero as stronger evidence of lack of fit. The theory is discussed in Smyth (2003), "Pearson's goodness of fit statistic as a score test statistic", Statistics and science: a Festschrift for Terry Speed.

In practice people usually rely on the asymptotic approximation of both to the chi-squared distribution - for a negative binomial model this means the expected counts shouldn't be too small. Smyth notes that the Pearson test is more robust against model mis-specification, as you're only considering the fitted model as a null without having to assume a particular form for a saturated model. I've never noticed much difference between them.

You may want to reflect that a significant lack of fit with either tells you what you probably already know: that your model isn't a perfect representation of reality. You're more likely to be told this the larger your sample size. Perhaps a more germane question is whether or not you can improve your model, & what diagnostic methods can help you.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.