Background: I've been tasked with creating a rating model by Peril using GLMs. It's commercial lines property, so the data is pretty sparse. The carriers have been asking for Premiums by peril, so we're going with it regardless if it's a better model than a single pure premium model. I also have to use a GLM because it needs to be a closed formula multiplicative model that can be piped into SQL
My data goes back 15 years, but only 2011-2017 has complete information on some variables so I only use those years. After doing all the necessary scrubbing, I'm sitting at only 50,000 policies with about 6.5% having an incurred claim.
Split by Peril: Peril1 has 900 claims. Of those 850 have 1 claim, 45 have 2 claims and 5 have 3 claims
Peril2 and Peril 3 have 1500 and 800 claims respectively with a similar claim count breakout as Peril1
I split the data using a random partition of 70% train, 30% test, and leave 2017 as a validation set. The same breakout percentages are reserved.
For the frequency models, they're in the R format:
glm(formula = Count_Peril1 ~ Variables, family = poisson(link = "log), offset = log(Exposure), data = data.train)
I log all of the continuous variables. My main measure of performance is the Gini. For frequency, that's Counts on the X axis and Exposure on the Y axis. The code I use to create the Gini is this:
o <- with(f.model1, order(prediction)) x <- with(data.test, cumsum(Count_Peril1[o]) / sum(Count_Peril1[o])) y <- with(data.test, cumsum(Exposure[o]) / sum(Exposure[o])) dx <- x[-1] - x[-length(x)] h <- (y[-1] + y[-length(y)]) / 2 gini.peril1 <- 2*(.5 - sum(h*dx))
I also use AIC to compare models. I've been using deviance ratio as a replacement for R^2, ie how much of the model is actually explained by the data.
I create the deviance ratio by:
deviance <- 1-(model.peril1$deviance / model.peril1$null.deviance)
I measure severity with a glm, but target variable is Inc_Peril1, family is Gamma, offset is log(Count_Peril1) and subset of Count_Peril1 > 0
The gini is calculated similarly as above, but x axis is Count_Peril1 and y axis is Incurred_Peril1.
Problem: I feel like my models are horrible and I don't know how/where to improve them first. The Q-Q plots suggest I'm using the wrong distribution. I've tried using negative binomial with various thetas, but that didn't seem to work.
Also, when I create the null model just looking at the intercept, the Gini is much higher than when I include any variables. The AIC and deviance are worse though. Not sure why that's the case.
While testing different variables, I check the summary to see if they're statistically significant. Then I'll look at the Gini, AIC, and deviance. I'll add/remove variables checking for improvements. Once I start only marginally increasing the Gini, I'll do an anova chi squared test to determine which models are best. Gini varies from .08-.22, which sounds pretty horrible, but I have no benchmark. When I run the summary plots, they all look pretty bad.
Here's an example of Peril1's Frequency model
Where do I go from here to improve the model?