0
$\begingroup$

I am running a glm model with Poisson Regression and I wanted to know a couple of things.

1) If my model has overdispersion?

2) I am using this model for insurance purposes ~ that is for rating so can I model with all categorical variables / factors or do I need to convert it to numeric variables

3) How can I plot the exp(coefficients) for each variable in R

Here is the code :

rr1 = exp(coef(fit))
write.csv(rr1,file="Coeffmod.csv")

4) Also if someone can please tell me how to check the goodness of fit for the model?

Thank you so much!

Any answer is much appreciated!

Here are my outputs:

Call:
glm(formula = NYDF$Gr_Actual ~ NYDF$Gr_DTC + NYDF$Gr_Construction.Type + 
    NYDF$Gr_Constr.Year + NYDF$Gr_Constr.Year.Roof + NYDF$Gr_Sq..Footage + 
    NYDF$Gr_Foundation.Basement.Percentage + NYDF$Gr_Num_stories + 
    NYDF$Gr_Num_Baths + NYDF$Number.Of.Fireplaces + NYDF$Gr_FireAlarm + 
    NYDF$Gr_Central.Air + NYDF$Gr_Full.Bath.Quality + NYDF$Gr_Structure.Type + 
    NYDF$Gr_Roof.Geometry.Type + NYDF$Gr_Roof.Covering.Type + 
    NYDF$Gr_Opening.Protection.Type + NYDF$Gr_Special.Loss.Settlement + 
    NYDF$Gr_ShortTerm.Rental + NYDF$Gr_Single.Occupancy + NYDF$Gr_Absentee.Landlord + 
    NYDF$Gr_Theft.Coverage + NYDF$Gr_Vacancy.Coverage + NYDF$Gr_Mold.Limit + 
    NYDF$Gr_Property.Occupancy + NYDF$Gr_Property.Usage + NYDF$Hurricane.Deductible + 
    NYDF$Gr_AOP.Deductible + NYDF$Coverage.A + NYDF$Gr_CovB + 
    NYDF$Gr_CovC + NYDF$Gr_CovD + NYDF$Gr_Garage.Type + NYDF$Coverage.L + 
    NYDF$Coverage.M + NYDF$Payment.Plan.Type + NYDF$Gr_Insurance.Scoring + 
    NYDF$Gr_Affiliation + NYDF$Gr_Seasonal.Surcharge.Factor + 
    NYDF$Gr_Secured.Community + NYDF$Gr_Additional.Amounts.Of.Insurance + 
    NYDF$Gr_Mechanical.Breakdown.Coverage + NYDF$Gr_Identity.Fraud.Coverage + 
    NYDF$Gr_Lapse.in.Coverage + NYDF$Gr_Prop.manager + NYDF$Gr_HomeFeatures + 
    NYDF$Gr_Water.Backup.Coverage, family = poisson(link = "log"), 
    data = NYDF)

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for poisson family taken to be 1)

    Null deviance: 13608.9  on 7970  degrees of freedom
Residual deviance:  9808.9  on 7776  degrees of freedom
  (737 observations deleted due to missingness)
AIC: 10780

Number of Fisher Scoring iterations: 17
$\endgroup$
2
  • $\begingroup$ Most of my experience is with life insurance underwriting, but in general you will need to categorize variables at some point to get this into a rating manual. That is unless you have a PC-based rating manual into which you can export a score. Send me an email if you want to discuss further. Looks like yu are on the borderline for adequate data for this task. 200 variable categories with only 7970 cases is unfortunately likely to leave a lot of ambiguity as far as risk. How may claims were there in that series? $\endgroup$
    – DWin
    Oct 14, 2016 at 4:03
  • 1
    $\begingroup$ @MrFlick, this is a bit close to the borderline (Q1 is a combined stats/R question; Q2 I don't really know; Q3 is strictly an R question; Q4 is mostly stats). If you think it's appropriate, you should vote to close/migrate ... $\endgroup$
    – Ben Bolker
    Nov 9, 2016 at 22:23

1 Answer 1

1
$\begingroup$

1) If my model has overdispersion?

Note this line of the summary:

Residual deviance:  9808.9  on 7776  degrees of freedom

Under Poisson variance, the residual deviance is $\chi^2$ distributed, so the expected value of the deviance is equal to the degrees of freedom (7776). (Resid dev/res df)=1.26, so we have a small to moderate level of overdispersion (i.e., resid deviance is 26% higher than expected), but with a data set this size, that's highly statistically significant: pchisq(9809,df=7776,lower.tail=FALSE) is $1.27 \times 10^{-51}$ ...

2) I am using this model for insurance purposes ~ that is for rating so can I model with all categorical variables / factors or do I need to convert it to numeric variables

Sorry, can't help you there.

3) How can I plot the exp(coefficients) for each variable in R

This is an R question, not a stats question, but plot(exp(coef(fit))) should do it! You might want to check out the dotwhisker package for prettier, easier coefficient plots ...

4) Also if someone can please tell me how to check the goodness of fit for the model?

(1) In one sense, overdispersion is a measure of lack of fit (see answer to Q1), so by that definition your model does not fit well. (2) Alternatively, there is a whole bestiary of pseudo-$R^2$ measures (see e.g. the nagelkerke() function of the rcompanion package) which you can use to compute "relative measures among similar models indicating how well the model explains the data". (3) The plot() method for a glm object will plot a series of diagnostic plots (residuals vs fitted, Q-Q, scale-residual, measures of influence ...)

P.S. All of those NYDF$ in your model formula are redundant, they just make things uglier and make downstream processing (e.g. predicting responses from new sets of covariates) impossible. Also, if you want to regress a response on all other variables in the data frame, try just Gr_Actual ~ . ...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.