3
$\begingroup$

We are working on a project where we want to find the pure premium for which we first want to fit Poisson regression. Data Used: data(dataCars) from insauranceData package in R We have an offset term exposure which takes value between 0 and 1 when I fit my model then I am getting rates between 0 and 1 but when I calculate the original rates to compare my prediction, there are values in 365claims/year (since few observations had claimed within 0.0027 exposure time) So is my interpretation of no of claims using Poisson GLM incorrect or is it wrong to compare the rates calculated using the model and calculated from the original data ? What is the right method to go about it? Please Help

Following are the codes attached:

library(insuranceData)
library(forcats)
library(caret)

data(dataCar)
data1 <- dataCar
#Data Cleaning & Pre-processing
data2 <- unique(data1)
data3 <- data2[data2$veh_value > quantile(data2$veh_value, 0.0001),] 
data4 <- data3[data3$veh_value < quantile(data3$veh_value, 0.999), ]
#Regrouping vehicle categories
top9 <- c('SEDAN','HBACK','STNWG','UTE','TRUCK','HDTOP','COUPE','PANVN','MIBUS')
data4$veh_body <- fct_other(data4$veh_body, keep = top9, other_level = 'other')
#Converting catagorical variables into factors
names <- c('veh_body' ,'veh_age','gender','area','agecat')
data4[,names] <- lapply(data4[,names] , factor)
str(data4)
##data partition - original data
data <- data4
data_partition <- createDataPartition(data$numclaims, times = 1,p = 0.8,list = FALSE)
str(data_partition)
training <- data[data_partition,]
testing  <- data[-data_partition,]


##data partition - re-sampled data
data <- data4
data_partition <- createDataPartition(data$numclaims, times = 1,p = 0.8,list = FALSE)
str(data_partition)
training <- data[data_partition,]
testing  <- data[-data_partition,]


#Number of Claims(Orignal Data)
table(training$numclaims)
ggplot(training,aes(numclaims))+geom_bar(fill="deepskyblue4")+
  geom_text(aes(label = ..count..), stat = "count", vjust = -0.3, colour = "black")+
  xlab("No of Claims")+
  ylab("Count of Claims")+
  ggtitle("Histogram of Number of Claims(in given exposure)")+
  theme(plot.title = element_text(hjust=0.5))+
  theme_update()

#Poisson model with offset
poissonglm <- glm(numclaims ~veh_value+veh_body+veh_age+gender+ area+ agecat+offset(log(exposure)),data=training, family = "poisson")
summary(poissonglm)

predict(poissonglm,newdata=data,type="response")

Our Models Predicts the Rate but since now the exposure term is taken care of,so the Rates are No of claims per year so Rate=No of Claims (here)?

I want my predictions to be in some range of counts (i.e 0 1 2 4) which I can compare with the number of claims in my original data.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

"when I fit my model then I am getting rates between 0 and 1 but when I calculate the original rates to compare my prediction, there are values in 365claims/year"

When I try to reproduce your problem, then I find no predicted values above 365. Or maybe you are looking at some different result?

histogram of results

I want my predictions to be in some range of counts (i.e 0 1 2 4) which I can compare with the number of claims in my original data.

The predictions are for the individual means or rates. Based on the rates you can make predictions for the number of counts in a particular bin.

The expected number of counts in a particular bin is equal to the sum of probability over all individuals

$$E[\text{total number of cases } x_i = k] = \sum_{i=1}^n P(x_i = k)$$

code example:

means = predict(poissonglm,newdata=data,type="response")

probs_individual = sapply(0:10, function(x) dpois(x,means))
expected = colSums(probs_individual)
print(round(expected))

This gives the following table

Number of claims   0     1      2     3   
Expected counts    62681 4440   228   0

Your original data has more often 3 or more claims than predicted. This might be because the true distribution is not Poisson and can be overdispersed (or the true data is Poisson distributed, but you can't describe the outliers well with your regressors/predictors).

Improve this answer
$\endgroup$
10
  • $\begingroup$ I am getting it as 365 when I look for rates of original data which is calculated by doing numcalims/exposure. (if I am not wrong) And also from, where did we end up with that formula (regarding The expected number of counts in a particular bin ), could you refer any book or article about the same. and actually I want the number of claims of an individual as in ,how many times would a particular person claim based on the values of predictor variable. or is it something which we cannot find? $\endgroup$
    – Aniket Kanse
    Nov 25, 2022 at 11:37
  • $\begingroup$ @AniketKanse the Poisson regression models the number of claims as a Poisson distributed random value computes the rate or average of that value. For instance for the 35729-th person this might be $\hat\lambda_{\text{individual number 35729}} = 0.4$ you can use that value to estimate the distribution for the number of claims that the individual makes. The formula for the distribution is a Poisson distribution. The expectation for the total is the sum of the expectation for the individuals. If these individuals are assumed independent then you can sum it. $\endgroup$
    – Sextus Empiricus
    Nov 25, 2022 at 12:20
  • $\begingroup$ So I can find the distribution but not the number of claims for say 35729-th person right? like cant I say that he would claim Once in 2.5yrs (i.e 1/0.4 since its rate of claiming is 0.4/year) also I am so sorry but I didn't get the expectation part since in the formulae which you gave had sum of the "probability" that he would claim k number of time $\endgroup$
    – Aniket Kanse
    Nov 25, 2022 at 13:38
  • $\begingroup$ It's like rolling a 6 sided dice ten times. You can say that you can get a 'six' for 1/6-th of the time, and in ten rolls on average 1 4/6 sixes. The probability to roll zero sixes is 1-(5/6)^10, and similarly you can compute the probability for one six, two sixes, etc. Then, with those probabilities for the individual, you can compute how often on average a dice that is being rolled ten times will roll a specific number of sixes. That average/frequency/probability can be used to estimate the expectation for the number of times you get a particular outcome. $\endgroup$
    – Sextus Empiricus
    Nov 25, 2022 at 14:15
  • $\begingroup$ Okay This made sense,......Though, So what is the use of Poisson Regression as we cant really "predict" the number of claims of an individual (i.e I cannot firmly say that ith individual is claiming 1 time in a year or etc ) $\endgroup$
    – Aniket Kanse
    Nov 25, 2022 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.