2
$\begingroup$

I'm using a data set of an insurance company, and I want to model the number of claims (counts) as a dependent variable (number of insurance claims, nb_sinistre in this data set). In R I use a glm with a Poisson distribution (link = log). Not every observation is observed for the same period. The exposure of a observation is between 0 and 1 (1 = one year).

A sample of the data set "TOBETESTEDR":

  nb_sinistre        nb NUCAPIDX CDCHINCA CDTAKEOB nb_sinistreAsRate
1           0 0.2465753 294624.0        1        3                 0
2           0 0.2465753  20000.0        3        3                 0
3           0 0.2739726 245520.0        1        3                 0
4           0 0.4684932 297099.8        4        3                 0
5           0 0.4684932  63361.5        3        3                 0
6           0 0.4794521 216000.0        1        3                 0

I put exposure ('nb' in this data set) as an offset in the formula. This is shown here why: When to use an offset in a Poisson regression?

For1 <- as.formula(nb_sinistre ~ NUCAPIDX + CDCHINCA + CDTAKEOB + offset(log(nb)))
Pois1 <- glm(For1, data = TOBETESTEDR, family = poisson(link = "log"))
summary(Pois1)

This returns:

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept) -6.256e+00  1.354e+00  -4.622  3.8e-06 ***
NUCAPIDX     7.132e-07  2.716e-07   2.626  0.00863 ** 
CDCHINCA     1.453e-01  5.538e-02   2.624  0.00868 ** 
CDTAKEOB     6.857e-01  4.472e-01   1.533  0.12520

Now I want to know whether I will obtain the same result, but by using the exposure not as an offset, but just change the dependent variable into a ratio itself. Reading this it seems to me it must be possible: How is it possible that Poisson GLM accepts non-integer numbers? So divide the number of claims (count) by the exposure and use this as the dependent variable:

TOBETESTEDR$nb_sinistreAsRate <- TOBETESTEDR$nb_sinistre / TOBETESTEDR$nb

For2 <- as.formula(nb_sinistreAsRate ~ NUCAPIDX + CDCHINCA + CDTAKEOB) 
Pois2 <- glm(For2 , data = TOBETESTEDR, family = poisson(link = "log"))
summary(Pois2)

This returns

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept) -7.659e+00  1.176e+00  -6.514 7.31e-11 ***
NUCAPIDX     6.275e-07  1.684e-07   3.726 0.000195 ***
CDCHINCA     2.688e-01  3.373e-02   7.971 1.57e-15 ***
CDTAKEOB     1.130e+00  3.899e-01   2.898 0.003756 **

Which is not exactly the same as the first method. Why are those two not the same?

Thanks for your help!!
KR
Leonard

$\endgroup$
1
$\begingroup$

I believe the issue is coming in because you are representing events as rates where you have 0 events.

Let's walk through how the offset is added (following the notation from the linked answer):

A poisson regression of the form: $log(\mu_x) = \beta_0 + \beta_1x$ is adjusted for different measurement times by correcting the expected counts ($\mu_x$) for the exposure time ($t_x$): $$log({\mu_x\over t_x}) = \beta_0 + \beta_1x$$

This is then simplified to introducing the offset by the rules of log transformations to give you: $$log(\mu_x) = log(t_x) + \beta'_0 + \beta'_1x$$

Note that the parameter $\mu_x$ can never be 0 because otherwise it's log is undefined (log(0) is forbidden).

So now let's take a look at what happens when you just calculate the rates of your model when you have 0 counts. For example take the first few rows of your data table: rows 1-3 have nb ~ 0.25 and rows 4-6 have nb ~ 0.47 but there are no counts for any of them. If you ratio these however all six rows have the same value all of a sudden: 0.

So the issue then is: zero is a difficult number to deal with because division and multiplication doesn't change the value.

So when you transformed your data by taking the ratio you actually slightly changed the data set that was being fit and that's why you have different answers.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ @Leonard, normally when the response is non-integer, glm in R will return warnings. I saw that you have a lot of zeros, so you should check nb is not all zeros. Patrick got it right, it's the zeros thats giving you problems $\endgroup$ – StupidWolf Oct 21 '19 at 20:54
  • $\begingroup$ @Patrick thanks for your explanation! So if I get it right, by dividing my outcome variable with the exposure, I will lose information in case the observed outcome is zero. More specific, for those observations I will lose the information 'exposure'. All observations with a zero observed outcome will then be weighted for one year. Thus, I have changed my data set, which gives me a different result. Correct? $\endgroup$ – Leonard Oct 22 '19 at 7:50
  • $\begingroup$ @StupidWolf indeed R is trowing warnings at me for every outcome variable that is not a discreet value. But I have read that should not be a problem (see link in my question). As you suggested I have checked the outcome variable. There are indeed a lot of zero's. But for this insurance product they said it was normal to have a fairly low claim-frequency. Maybe a ZIP is more appropriate here. $\endgroup$ – Leonard Oct 22 '19 at 7:54
  • $\begingroup$ ZIP is another thing altogether. If as you said, claims are low frequency, using the poisson is ok. My point is, the regression with offset works just fine, why do you want to regress against the rate? $\endgroup$ – StupidWolf Oct 22 '19 at 7:59
  • $\begingroup$ @StupidWolf, no reason at all to use rates instead of an offset :). Just curious to know why it was not the same! Actually I was testing whether R would accept non discreet outcome variables. To be fast I just divided the outcome variable with the exposure to obtain non discreet values. It was then that I observed the result was slightly different then using exposure as an offset. $\endgroup$ – Leonard Oct 22 '19 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.