0
$\begingroup$

I am fairly new to GLMs, and am currently practising and testing with an insurance dataset, after many tries, I am modeling the frequency (counting model of the number of claims) and I have several doubts:

Why there are variables that are significant in the quasipoisson but not in the poisson? Which distribution is better? Why does quasipoisson not show the AIC? Imagine that I have variables that are not significant but if they are included the residual deviance becomes lower, is it a good idea to include them in spite of the fact that they are not significant?

Call:
glm(formula = formula, family = poisson, data = BBDD_SIN)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.0437  -0.2712  -0.1605   0.1816   2.5796  

Coefficients:
                    Estimate Std. Error z value Pr(>|z|)    
(Intercept)         0.206505   0.020785   9.935  < 2e-16 ***
factor(agecat)2     0.056670   0.013373   4.238 2.26e-05 ***
factor(agecat)3     0.016077   0.013170   1.221 0.222188    
factor(agecat)4     0.106790   0.012752   8.374  < 2e-16 ***
factor(agecat)5    -0.045922   0.014480  -3.171 0.001517 ** 
factor(agecat)6    -0.038017   0.016196  -2.347 0.018907 *  
areaB              -0.046294   0.010021  -4.620 3.84e-06 ***
areaC              -0.011760   0.008691  -1.353 0.176033    
areaD              -0.068055   0.011974  -5.683 1.32e-08 ***
areaE              -0.059073   0.013429  -4.399 1.09e-05 ***
areaF               0.294737   0.010334  28.520  < 2e-16 ***
veh_age2           -0.021878   0.022446  -0.975 0.329718    
veh_age3            0.078486   0.020270   3.872 0.000108 ***
veh_age4           -0.167408   0.021156  -7.913 2.51e-15 ***
veh_value          -0.026882   0.005538  -4.854 1.21e-06 ***
genderM             0.057310   0.006312   9.079  < 2e-16 ***
veh_age2:veh_value  0.030768   0.008063   3.816 0.000136 ***
veh_age3:veh_value  0.028417   0.008266   3.438 0.000586 ***
veh_age4:veh_value  0.104698   0.011647   8.989  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for poisson family taken to be 1)

    Null deviance: 15148  on 78607  degrees of freedom
Residual deviance: 12384  on 78589  degrees of freedom
AIC: 184843

Number of Fisher Scoring iterations: 4

Quassipoisson

Call:
glm(formula = formula, family = quasipoisson(), data = BBDD_SIN)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.0437  -0.2712  -0.1605   0.1816   2.5796  

Coefficients:
                    Estimate Std. Error t value Pr(>|t|)    
(Intercept)         0.206505   0.008724  23.672  < 2e-16 ***
factor(agecat)2     0.056670   0.005613  10.097  < 2e-16 ***
factor(agecat)3     0.016077   0.005528   2.909  0.00363 ** 
factor(agecat)4     0.106790   0.005352  19.952  < 2e-16 ***
factor(agecat)5    -0.045922   0.006077  -7.556 4.20e-14 ***
factor(agecat)6    -0.038017   0.006798  -5.593 2.24e-08 ***
areaB              -0.046294   0.004206 -11.007  < 2e-16 ***
areaC              -0.011760   0.003648  -3.224  0.00127 ** 
areaD              -0.068055   0.005026 -13.541  < 2e-16 ***
areaE              -0.059073   0.005636 -10.481  < 2e-16 ***
areaF               0.294737   0.004337  67.952  < 2e-16 ***
veh_age2           -0.021878   0.009421  -2.322  0.02022 *  
veh_age3            0.078486   0.008507   9.226  < 2e-16 ***
veh_age4           -0.167408   0.008879 -18.853  < 2e-16 ***
veh_value          -0.026882   0.002324 -11.565  < 2e-16 ***
genderM             0.057310   0.002649  21.632  < 2e-16 ***
veh_age2:veh_value  0.030768   0.003384   9.092  < 2e-16 ***
veh_age3:veh_value  0.028417   0.003469   8.191 2.63e-16 ***
veh_age4:veh_value  0.104698   0.004888  21.418  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for quasipoisson family taken to be 0.1761547)

    Null deviance: 15148  on 78607  degrees of freedom
Residual deviance: 12384  on 78589  degrees of freedom
AIC: NA

Number of Fisher Scoring iterations: 4
```
$\endgroup$
1
  • $\begingroup$ It is best not to have too many questions in the same thread, & in particular not to mix substantively different questions. Your last stated question, "imagine that..." should be posted to a new thread. $\endgroup$ Mar 12, 2021 at 18:21

1 Answer 1

2
$\begingroup$

The Poisson distribution makes a very strong assumption that the variance is equal to the mean. This assumption doesn't typically hold in real world data. However, the Poisson GLM relies on that assumption in computing the standard errors. Thus, that assumption needs to hold for the inferences (i.e., p-values) to be valid. Since it doesn't typically, there is demand for strategies to deal with that.

One such strategy is to estimate the amount of over-(rarely, but possibly under)-dispersion, and simply multiply the Poisson GLM standard errors by that value. That will typically make the quasipoisson SEs larger and thus decrease the significance of the variables. In your case, you have underdispersion (you can see the estimated dispersion is 0.1761547. So when you multiply the original SEs by that fraction, they get smaller, and the p-values get lower. Thus, they are more significant in the quasipoisson model in this case.

This trick is a bit ad-hoc. The 'quasipoisson' distribution isn't quite a real distribution in this sense and doesn't have a real likelihood. As a result, there is no AIC.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.