I am using GLM to find the best features that determine the termination rate of a life insurance's income protection claim book. (Some background: if a person is injured or got sick, he can make claims and a monthly benefit will be paid as long as he is still sicked/injured. I am trying to predict what could influence his recovery and terminate the claim)
I have an industry expected rate which I want to incorporate into my model. The chosen model is GLM with Poisson regression. However, rates are calculated monthly hence we need to duplicate the data to allow for that. If the claim is terminated, the number of records would be the duration from month1 till the time it is terminated. If it is not, the number of records would be the duration from month1 till the time the benefits end. In this case, that is 24 months
For example, my original data would be:
Claim no Feature1 Feature2 Termination Ind 1 TRUE TRUE 1 2 FALSE TRUE 0
Then it should be transformed into (claim 1 is duplicated 3 times and claim 2 is duplicated 24 times)
Claim no Feature1 Feature2 Expected.Rte Month Term.Ind 1 TRUE TRUE 0.022 1 0 1 TRUE TRUE 0.025 2 0 1 TRUE TRUE 0.024 3 1 2 TRUE TRUE 0.022 1 0 2 TRUE TRUE 0.005 2 0 2 TRUE TRUE 0.014 3 0 ............................ 2 TRUE TRUE 0.04 24 0
Then the model is fitted with all the features above (I have about 20). Now I would like to find a way to estimate the model performance to select the best features.
I have the null model set as:
fit.null = glm(Term.Ind ~ offset(log(Expected.rate)), data, family = Poisson(link = log))
I built a function to do the following steps:
- Start with the null model - Add one feature - Run cross-validation - Read the delta value - Repeat the same step with the next feature - Then I will select the next feature
Results: I found the delta extract from cv.glm() seems to be very small (less than 2%) from the null model. If I add a new factor, the delta value reduces marginally and not very helpful to tell which factor is the best. Hence I think this could indicate that I should not use MSE to evaluate the performance of the model.
Another issue I found is the R squared from the null model is negative which suggest that my null model is even worse than the mean value.
Any suggestion on the method and the measure of accuracy would be greatly appreciated.