I am using a hurdle model (dist=negbin, link = logit) for a dataset with multiple explanatory variables, excessive zeros and overdispersion by both, zeros and count data.

The residual plots (pearson residuals against fitted values) are not very pleasing, datapoints are mostly clustered in the bottom left corner, so I tried transforming the response. Square root transformation led to much "better" residual plots. I have read, that count data should not be transformed, but this was with reference to glm models. But would it be an appropriate approach when using a hurdle model?

enter image description here

enter image description here

enter image description here

  • $\begingroup$ I doubt this is any better with a hurdle model. Can you paste in your residual plots? $\endgroup$ – gung May 5 '15 at 14:47
  • $\begingroup$ sorry for the late response, I've added residual plots but I guess transforming is not an option $\endgroup$ – Jamy May 12 '15 at 18:13

The hurdle (and zero-inflation) count regression models typically use a log-link for the count component of the model. So you already have a log-linear relationship between the expectation of the (latent) count component and the regressors already. Hence, further transformations are usually not necessary/applicable.

Furthermore, a sqrt-transformation of the response does not yield count data anymore, hence this does not make much sense for count data.

I see three possible actions for you:

  1. Do not look at just the residuals but also compare observed and expected frequencies for 0, 1, 2, ... counts. Often the fitted distribution is quite satisfactory even if the residuals are large - especially when the overdispersion is substantial (i.e., low theta in the negbin distribution).
  2. You have to consider either a different count distribution and/or improve the specification of the regressors in your model to obtain a better fit.
  3. If your counts are large enough, it may be possible to use a continuous rather than a discrete distribution. Thus, instead of the zero-truncated negative binomial distribution, you could use a zero-truncated normal distribution - either for the original response or after a possible sqrt-transformation etc.

Which of these is most appropriate depends on your data, sample size, and the kind of predictions you want to make.

  • $\begingroup$ Thanks very much for the detailed answers and for clarifying. $\endgroup$ – Jamy May 12 '15 at 17:56
  • $\begingroup$ On Nr. 3: What would "large enough" mean in this context (I've added a histgram of the response to the question)? Could this be implemented with the hurdle function or would another model type be reasonable? $\endgroup$ – Jamy May 12 '15 at 18:22
  • $\begingroup$ Your response values are huge. I would probably try to first look at a binary model for zero vs. greater (logit, probit, ...). And then you can try to model the positive values only, e.g., with a zero-truncated normal model or after log- or sqrt-transfornation. $\endgroup$ – Achim Zeileis May 12 '15 at 21:13
  • $\begingroup$ Thanks again!! I’ve failed fitting a zero-truncated normal model using the VGAM or truncreg package (Can you suggest other options?). However, it seems quite promising to model the log-tranformed positive counts with a simple lm-model, although there are still minor patterns. Could you recommend literature for the approach of using two different models for count data? $\endgroup$ – Jamy May 13 '15 at 9:52
  • $\begingroup$ I typically use truncreg or VGAM myself. Modeling "large" counts by a simple OLS regression with a log-transformed response is fairly common and covered in many textbooks. In econometrics textbooks, this is often discussed under "Limited Response Models". Personally, I like the book of Winkelmann & Boes "Analysis of Microdata" very much. But there are, of course, other good books as well. $\endgroup$ – Achim Zeileis May 13 '15 at 12:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.