# Using one or many training test - GLM Poisson

I'm newbie to GLM. I'm modeling frequence of accident from driver age, vehicle age, vehicle power... using GLM Poisson I create a training set and a test set. However, I notice for each time I run the model (so the training set changes), the coefficients change (and also the significance level). My question is that: should we change our training set several times or not? If yes, how to deal with the problem above.

I show there 2 examples (2 different training sets):

Call:
glm(formula = TPL ~ DrivAge + DrivGender + MaritalStatus + BonusMalus +
LicenceNb + PayFreq + JobCode + VehAge + VehClass + VehPower +
VehGas + VehUsage + Garage + Area + Region + Channel + Marketing,
family = quasipoisson, data = train)

Deviance Residuals:
Min       1Q   Median       3Q      Max
-0.7683  -0.4040  -0.3583  -0.3164   4.8324

Coefficients: (3 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept)                      -3.433879   0.804993  -4.266 2.01e-05 ***
DrivAge                           0.008009   0.003139   2.552  0.01073 *
DrivGenderM                      -0.025324   0.073180  -0.346  0.72931
MaritalStatusDivorced             0.397871   0.269480   1.476  0.13985
MaritalStatusMarried             -0.017458   0.084927  -0.206  0.83713
MaritalStatusSingle               0.280122   0.099957   2.802  0.00508 **
MaritalStatusWidowed             -0.384041   0.235483  -1.631  0.10294
BonusMalus                        0.011595   0.002348   4.939 7.94e-07 ***
LicenceNb                        -0.011739   0.050265  -0.234  0.81535
PayFreqHalf-yearly                0.010197   0.078224   0.130  0.89629
PayFreqMonthly                    0.116260   0.162018   0.718  0.47303
PayFreqQuarterly                  0.025511   0.140871   0.181  0.85629
JobCodeFarmer                     0.282743   0.294779   0.959  0.33749
JobCodeOther                     -0.090008   0.233286  -0.386  0.69963
JobCodePrivate employee           0.050860   0.179362   0.284  0.77675
JobCodePublic employee            0.094397   0.183370   0.515  0.60671
JobCodeRetailer                 -12.532778 283.850222  -0.044  0.96478
JobCodeRetiree                    0.177935   0.217747   0.817  0.41385
VehAge                           -0.021195   0.007865  -2.695  0.00705 **
VehClassCheaper                  -0.033183   0.110429  -0.300  0.76381
VehClassCheapest                 -0.144738   0.148141  -0.977  0.32857
VehClassExpensive                 0.224670   0.375376   0.599  0.54950
VehClassMedium                    0.113419   0.197041   0.576  0.56489
VehClassMedium high              -0.029006   0.288382  -0.101  0.91988
VehClassMedium low                0.029851   0.135363   0.221  0.82546
VehClassMore expensive           -0.115352   0.544292  -0.212  0.83217
VehClassMost expensive            0.397490   0.632560   0.628  0.52976
VehPowerP11                       0.138692   0.129379   1.072  0.28375
VehPowerP12                       0.096941   0.139545   0.695  0.48726
VehPowerP13                       0.134677   0.166905   0.807  0.41974
VehPowerP14                       0.249407   0.199613   1.249  0.21152
VehPowerP15                      -0.030085   0.308138  -0.098  0.92222
VehPowerP16                       0.393038   0.398141   0.987  0.32357
VehPowerP17                       0.320942   0.809079   0.397  0.69161
VehPowerP2                      -11.743427 907.484450  -0.013  0.98968
VehPowerP4                      -12.058187 573.089763  -0.021  0.98321
VehPowerP5                       -0.995431   0.594664  -1.674  0.09417 .
VehPowerP7                        0.344036   0.268571   1.281  0.20022
VehPowerP8                        0.379639   0.142235   2.669  0.00761 **
VehPowerP9                        0.173538   0.133922   1.296  0.19506
VehGasRegular                    -0.127739   0.078080  -1.636  0.10186
VehUsageProfessional              0.299550   0.225092   1.331  0.18328
VehUsageProfessional run          0.576924   0.295204   1.954  0.05068 .
GarageClosed zbox                 0.213356   0.096891   2.202  0.02768 *
GarageOpened collective parking   0.047413   0.116148   0.408  0.68312
GarageStreet                      0.231024   0.122183   1.891  0.05867 .
AreaA12                          -0.161991   1.237855  -0.131  0.89588
AreaA2                           -0.500636   0.723129  -0.692  0.48875
AreaA3                           -0.546334   0.722099  -0.757  0.44931
AreaA4                           -0.223314   0.725155  -0.308  0.75812
AreaA5                           -0.352101   0.719116  -0.490  0.62440
AreaA6                           -0.568578   0.774222  -0.734  0.46273
AreaA7                           -0.414094   0.727111  -0.570  0.56902
AreaA8                           -0.375806   0.736787  -0.510  0.61002
AreaA9                           -0.230268   0.725615  -0.317  0.75099
RegionParis area                  0.182765   0.122177   1.496  0.13470
RegionSouth West                 -0.472009   0.586351  -0.805  0.42084
ChannelB                          0.142026   0.248928   0.571  0.56831
ChannelL                         -0.139609   0.082359  -1.695  0.09007 .
MarketingM2                             NA         NA      NA       NA
MarketingM3                             NA         NA      NA       NA
MarketingM4                             NA         NA      NA       NA
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for quasipoisson family taken to be 1.012757)

Null deviance: 5160.1  on 13503  degrees of freedom
Residual deviance: 5037.1  on 13444  degrees of freedom
(28055 observations deleted due to missingness)
AIC: NA

Number of Fisher Scoring iterations: 13


And:

Call:
glm(formula = TPL ~ DrivAge + DrivGender + MaritalStatus + BonusMalus +
LicenceNb + PayFreq + JobCode + VehAge + VehClass + VehPower +
VehGas + VehUsage + Garage + Area + Region + Channel + Marketing,
family = quasipoisson, data = train)

Deviance Residuals:
Min       1Q   Median       3Q      Max
-0.6924  -0.3969  -0.3545  -0.3144   4.7259

Coefficients: (3 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept)                      -3.758964   0.805871  -4.664 3.12e-06 ***
DrivAge                           0.008306   0.003166   2.623  0.00871 **
DrivGenderM                      -0.059227   0.072369  -0.818  0.41314
MaritalStatusDivorced             0.237533   0.286279   0.830  0.40671
MaritalStatusMarried              0.069704   0.083210   0.838  0.40222
MaritalStatusSingle               0.249467   0.101559   2.456  0.01405 *
MaritalStatusWidowed             -0.110189   0.204761  -0.538  0.59049
BonusMalus                        0.009858   0.002356   4.183 2.89e-05 ***
LicenceNb                        -0.006905   0.050557  -0.137  0.89137
PayFreqHalf-yearly                0.027229   0.078711   0.346  0.72940
PayFreqMonthly                    0.198096   0.157291   1.259  0.20790
PayFreqQuarterly                 -0.020713   0.145221  -0.143  0.88658
JobCodeFarmer                     0.565334   0.295674   1.912  0.05590 .
JobCodeOther                      0.213664   0.238004   0.898  0.36934
JobCodePrivate employee           0.210541   0.191649   1.099  0.27197
JobCodePublic employee            0.331952   0.194939   1.703  0.08862 .
JobCodeRetailer                   1.000891   0.612073   1.635  0.10202
JobCodeRetiree                    0.366832   0.227306   1.614  0.10659
VehAge                           -0.014329   0.007741  -1.851  0.06420 .
VehClassCheaper                  -0.013083   0.111105  -0.118  0.90627
VehClassCheapest                 -0.016907   0.149586  -0.113  0.91001
VehClassExpensive                 0.041037   0.390048   0.105  0.91621
VehClassMedium                    0.177040   0.186507   0.949  0.34252
VehClassMedium high              -0.436590   0.310360  -1.407  0.15953
VehClassMedium low               -0.068154   0.137396  -0.496  0.61987
VehClassMore expensive           -0.810244   0.688764  -1.176  0.23947
VehClassMost expensive            0.060609   0.704527   0.086  0.93145
VehPowerP11                       0.152053   0.130469   1.165  0.24387
VehPowerP12                       0.148002   0.140920   1.050  0.29362
VehPowerP13                       0.196015   0.168605   1.163  0.24502
VehPowerP14                       0.489279   0.198917   2.460  0.01392 *
VehPowerP15                       0.210457   0.297820   0.707  0.47979
VehPowerP16                       0.535802   0.422429   1.268  0.20468
VehPowerP17                      -0.089318   1.062724  -0.084  0.93302
VehPowerP2                      -10.935041 345.147440  -0.032  0.97473
VehPowerP4                      -11.316674 344.768434  -0.033  0.97382
VehPowerP5                       -0.888187   0.513750  -1.729  0.08386 .
VehPowerP7                        0.291672   0.260037   1.122  0.26203
VehPowerP8                        0.344021   0.139878   2.459  0.01393 *
VehPowerP9                        0.094111   0.133048   0.707  0.47937
VehGasRegular                    -0.094493   0.077936  -1.212  0.22537
VehUsageProfessional              0.380818   0.218267   1.745  0.08105 .
VehUsageProfessional run          0.565450   0.305062   1.854  0.06382 .
GarageClosed zbox                 0.181757   0.096416   1.885  0.05943 .
GarageOpened collective parking   0.095442   0.113857   0.838  0.40190
GarageStreet                      0.247142   0.119960   2.060  0.03940 *
AreaA12                          -0.032918   1.228840  -0.027  0.97863
AreaA2                           -0.458926   0.717346  -0.640  0.52234
AreaA3                           -0.451137   0.716336  -0.630  0.52885
AreaA4                           -0.243743   0.720159  -0.338  0.73502
AreaA5                           -0.277866   0.713694  -0.389  0.69703
AreaA6                           -0.517108   0.775208  -0.667  0.50475
AreaA7                           -0.302781   0.724004  -0.418  0.67581
AreaA8                           -0.048111   0.731303  -0.066  0.94755
AreaA9                           -0.102986   0.722589  -0.143  0.88667
RegionParis area                  0.173735   0.122637   1.417  0.15660
RegionSouth West                 -0.838719   0.709602  -1.182  0.23724
ChannelB                          0.262125   0.244031   1.074  0.28278
ChannelL                         -0.109997   0.081158  -1.355  0.17533
MarketingM2                             NA         NA      NA       NA
MarketingM3                             NA         NA      NA       NA
MarketingM4                             NA         NA      NA       NA
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for quasipoisson family taken to be 0.9951371)

Null deviance: 5123.8  on 13616  degrees of freedom
Residual deviance: 5008.5  on 13557  degrees of freedom
(27942 observations deleted due to missingness)
AIC: NA

Number of Fisher Scoring iterations: 12


I'd suggest you to invest some more time in your models. Here are some issues that might be partly responsible for the unstable results:

1) Throwing away 2/3 of rows due to missing values is very bad practice. Try to retain these lines.

2) Why investing just one parameter in variables like DrivAge but many more for variables like area or power? The decision on how many parameters to use should not be based on the variable type of the data.

3) Why no interactions?

4) Three regressors are perfectly correlated with the others. That should not be happen.