1
$\begingroup$

So I'm trying to fit a hurdle model with the count distribution as negative binomial. I get the following outputs for assuming negative binomial and poisson:

> hurdle(degree ~ dc, data = data, dist = "negbin")

Call:
hurdle(formula = degree ~ dc, data = data, dist = "negbin")

Count model coefficients (truncated negbin with log link):
(Intercept)           dc  
     0.2428       0.1035  
Theta = 0.8815 

Zero hurdle model coefficients (binomial with logit link):
(Intercept)           dc  
    -0.8512       0.1649

> hurdle(degree ~ dc, data = data, dist = "poisson")

Call:
hurdle(formula = degree ~ dc, data = data, dist = "poisson")

Count model coefficients (truncated poisson with log link):
(Intercept)           dc  
    0.68283      0.08584  

Zero hurdle model coefficients (binomial with logit link):
(Intercept)           dc  
    -0.8512       0.1649  

From a regression in python based on estimates of mean of non zero data vs. regressor, I get:

m = 0.08374289, b = 0.7132967

Which is far from what the negative binomial estimates, but the Poisson gets it pretty close. However a vuong test tells me that the negative binomial is far better:

> vuong(mod_pois, mod_nb_hurdle)
Vuong Non-Nested Hypothesis Test-Statistic: -114.0873 
(test-statistic is asymptotically distributed N(0,1) under the
 null that the models are indistinguishible)
in this case:
model2 > model1, with p-value 0 

The non-zero data is overdispersed, but it looks like the variance is constant*mean, so I know Poisson shouldn't be used, but why is a Poisson hurdle so much better at predicting log(expected value)?

$\endgroup$
  • $\begingroup$ It is unclear to me what you are asking. What are the estimates "m" and "b" from Python? The mean or the intercept and the slope or...? And what link functions have been used in the Python estimates? For the Vuong test mod_pois and mod_nb_hurdle have not been defined but quite likely the two models are nested, making the Vuong test not the right kind of test for the question. A reproducible example and a better explanation what you think should be correct might make it possible to give a useful answer. $\endgroup$ – Achim Zeileis Jan 29 '16 at 7:36
1
$\begingroup$

You cannot use Vuong (a likelihood-ratio-based test) to compare models with with different likelihood functions. Poisson and Neg Bin have different likelihood functions.

$\endgroup$
  • $\begingroup$ Can you provide a reference for your statement as i do not think it is true that Vuong is likelihood based. $\endgroup$ – mdewey Sep 2 '17 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.