Background: I've been tasked with creating a rating model by Peril using GLMs. It's commercial lines property, so the data is pretty sparse. The carriers have been asking for Premiums by peril, so we're going with it regardless if it's a better model than a single pure premium model. I also have to use a GLM because it needs to be a closed formula multiplicative model that can be piped into SQL

My data goes back 15 years, but only 2011-2017 has complete information on some variables so I only use those years. After doing all the necessary scrubbing, I'm sitting at only 50,000 policies with about 6.5% having an incurred claim.

Split by Peril: Peril1 has 900 claims. Of those 850 have 1 claim, 45 have 2 claims and 5 have 3 claims

Peril2 and Peril 3 have 1500 and 800 claims respectively with a similar claim count breakout as Peril1

I split the data using a random partition of 70% train, 30% test, and leave 2017 as a validation set. The same breakout percentages are reserved.

For the frequency models, they're in the R format:

glm(formula = Count_Peril1 ~ Variables,
       family = poisson(link = "log),
       offset = log(Exposure),
       data = data.train)

I log all of the continuous variables. My main measure of performance is the Gini. For frequency, that's Counts on the X axis and Exposure on the Y axis. The code I use to create the Gini is this:

o <- with(f.model1, order(prediction))
x <- with(data.test, cumsum(Count_Peril1[o]) / sum(Count_Peril1[o]))
y <- with(data.test, cumsum(Exposure[o]) / sum(Exposure[o]))
dx <- x[-1] - x[-length(x)]
h <- (y[-1] + y[-length(y)]) / 2
gini.peril1 <- 2*(.5 - sum(h*dx))

I also use AIC to compare models. I've been using deviance ratio as a replacement for R^2, ie how much of the model is actually explained by the data.

I create the deviance ratio by:

deviance <- 1-(model.peril1$deviance / model.peril1$null.deviance)

I measure severity with a glm, but target variable is Inc_Peril1, family is Gamma, offset is log(Count_Peril1) and subset of Count_Peril1 > 0

The gini is calculated similarly as above, but x axis is Count_Peril1 and y axis is Incurred_Peril1.

Problem: I feel like my models are horrible and I don't know how/where to improve them first. The Q-Q plots suggest I'm using the wrong distribution. I've tried using negative binomial with various thetas, but that didn't seem to work.

Also, when I create the null model just looking at the intercept, the Gini is much higher than when I include any variables. The AIC and deviance are worse though. Not sure why that's the case.

While testing different variables, I check the summary to see if they're statistically significant. Then I'll look at the Gini, AIC, and deviance. I'll add/remove variables checking for improvements. Once I start only marginally increasing the Gini, I'll do an anova chi squared test to determine which models are best. Gini varies from .08-.22, which sounds pretty horrible, but I have no benchmark. When I run the summary plots, they all look pretty bad.

Here's an example of Peril1's Frequency model

Model Summary (Gini of .12): enter image description here

Q-Q: https://i.imgur.com/CPZRGxT.png

Residual Plot: enter image description here

Cooks Distance: enter image description here

Where do I go from here to improve the model?


closed as too broad by Glen_b Apr 15 at 13:44

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ This question appears overly broad for our format ("where do I go from here" is vague, and invites speculation); it's also unclear quite what you need. $\endgroup$ – Glen_b Apr 15 at 13:46