How to evaluate goodness of fit for negative binomial regression I'm trying to fit a model estimating waiting time using negative binomial regression, but I'm not sure how to assess the goodness of fit for my model. I would like to compare the negative binomial model to a Poisson model. I have approximately $4,000$ data points.  Any suggestions?
Thanks!
 A: Generally speaking, a good fitting model means does a good job generalizing to data not captured in your sample. A good way to mimic this is through cross-validation (CV). To do this, you subset your data into two parts: a testing data set and a training data set. Based on your sample size, I would recommend randomly putting 70% of your data into a testing data set and the remaining 30% in a training data set.
Now, build both the Poisson model and the negative binomial model based on your training data set. Calculate the predicted values for the data in your testing data set and compare it to the actual values in the following way:
$\sum_{i=1}^{n_2} (Y_i - \hat{Y}_i)^2$
where $n_2$ is the sample size of your training data set, $Y_i$ is the actual value of the dependent variable, and $\hat{Y}_i$ is the predicted value of the dependent variable.
Whichever model provides a lower value for the above expression is the preferred model. 
Now, there is a modification of this called k-folds CV. What it will do is split your data into $k$ approximately equal subsets (called "fold") and will predict each fold using the remaining folds as training data. Setting $k=4$ seems reasonable to me.
The relevant R function for this is cv.glm() in the boot package. More information here: http://stat.ethz.ch/R-manual/R-patched/library/boot/html/cv.glm.html
A: I would suggest to use approaches such as the Akaike information criterion or Bayesian information criterion and compare the returned values of your two models (GLM vs. NBR).
Also, using cross-validation to see which model performs worse could be an option and is commonly used, at least to get an impression about how a learned model performs.
A: So, if you just want to know if your fit is significant, you can compute the p-value.
First, find a good metric for your problem. For distributions a typically used is the Kolmogorov-Smirnov distance: $KS(f,g)=max|f(x)-g(x)|$. Now, call $E$ the cdf of your data and $P$ the analytical cdf of your fit, then
$KS_0=KS(E,P)$.
Now, we want to compute the probability of obtaining a $KS>KS_0$, given that we assume that your fit is correct. We can easily do that by sampling $n$ times a set of points of size 4000 from your fitted distribution; then we fit the set, and we compute the $KS$ between the sampled set and the fit of the sampled. The p-value is simply the proportion of the $n$ sets where this $KS>KS_0$.
If the resulted p-value$>0.05$ (or some significance level that you need to set) then your data is compatible with your fit.
