I am estimating a hurdle model with a binomial (first stage) and truncated poisson distribution (second stage). The results look fine but I have a very large value of theta (greater than 10000). I understand that theta represents the overdispersion of the count data with respect to a Poisson distribution but I still wonder if such a large value says anything about the (maybe bad) fit of the model. From my research it doesn't seem to matter for fit, but I'd be great if someone could confirm this or provide a better explanation for theta.



Usually such a large theta reflects that there is simply no overdispersion in the data, possibly even some underdispersion.

As a simple example consider the following artificial Poisson-distributed regression model:

d <- data.frame(x = runif(500))
d$y <- rpois(500, exp(0 + 2 * d$x))

Thus, this is a plain Poisson GLM with intercept 0 and slope 2. Of course, these coefficients could also be recovered from a negative binomial hurdle, just by estimating more coefficients than necessary (count theta plus zero hurdle coefficients).

m1 <- hurdle(y ~ x, data = d, dist = "negbin")
coef(m1, model = "count")
## (Intercept)           x 
## -0.01399583  2.06624937 
##    count 
## 69.27435 

Thus, the coefficients are very close to their true values and theta = 69.3 is already rather close to infinity (= Poisson distribution).

If we now make the data somewhat more underdispersed by cutting only very few extreme counts, theta can go up extremely: First, we just omit a single observation, the one with the maximum y = 17:

hurdle(y ~ x, data = d, dist = "negbin", subset = y < max(y))$theta
##    count 
## 32186.64 

Omitting all y > 10 (i.e., nine observations), theta becomes even more extreme:

hurdle(y ~ x, data = d, dist = "negbin", subset = y <= 10)$theta
##    count 
## 343701.2 

Thus, the extreme theta reflects that you don't need a negative binomial model here. Possibly a Poisson distribution fits well enough, if the underdispersion is not too pronounced. A useful graphical check whether the Poisson distribution is an ok fit for your data is the rootogram, see Kleiber & Zeileis (2016, The American Statistician, doi:10.1080/00031305.2016.1173590).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.