Use the offset or use rates as dependent variable in Poisson regression

Question

I'm using a data set of an insurance company, and I want to model the number of claims (counts) as a dependent variable (number of insurance claims, nb_sinistre in this data set). In R I use a glm with a Poisson distribution (link = log). Not every observation is observed for the same period. The exposure of a observation is between 0 and 1 (1 = one year).

A sample of the data set "TOBETESTEDR":

  nb_sinistre        nb NUCAPIDX CDCHINCA CDTAKEOB nb_sinistreAsRate
1           0 0.2465753 294624.0        1        3                 0
2           0 0.2465753  20000.0        3        3                 0
3           0 0.2739726 245520.0        1        3                 0
4           0 0.4684932 297099.8        4        3                 0
5           0 0.4684932  63361.5        3        3                 0
6           0 0.4794521 216000.0        1        3                 0

I put exposure ('nb' in this data set) as an offset in the formula. This is shown here why: When to use an offset in a Poisson regression?

For1 <- as.formula(nb_sinistre ~ NUCAPIDX + CDCHINCA + CDTAKEOB + offset(log(nb)))
Pois1 <- glm(For1, data = TOBETESTEDR, family = poisson(link = "log"))
summary(Pois1)

This returns:

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept) -6.256e+00  1.354e+00  -4.622  3.8e-06 ***
NUCAPIDX     7.132e-07  2.716e-07   2.626  0.00863 ** 
CDCHINCA     1.453e-01  5.538e-02   2.624  0.00868 ** 
CDTAKEOB     6.857e-01  4.472e-01   1.533  0.12520

Now I want to know whether I will obtain the same result, but by using the exposure not as an offset, but just change the dependent variable into a ratio itself. Reading this it seems to me it must be possible: How is it possible that Poisson GLM accepts non-integer numbers? So divide the number of claims (count) by the exposure and use this as the dependent variable:

TOBETESTEDR$nb_sinistreAsRate <- TOBETESTEDR$nb_sinistre / TOBETESTEDR$nb

For2 <- as.formula(nb_sinistreAsRate ~ NUCAPIDX + CDCHINCA + CDTAKEOB) 
Pois2 <- glm(For2 , data = TOBETESTEDR, family = poisson(link = "log"))
summary(Pois2)

This returns

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept) -7.659e+00  1.176e+00  -6.514 7.31e-11 ***
NUCAPIDX     6.275e-07  1.684e-07   3.726 0.000195 ***
CDCHINCA     2.688e-01  3.373e-02   7.971 1.57e-15 ***
CDTAKEOB     1.130e+00  3.899e-01   2.898 0.003756 **

Which is not exactly the same as the first method. Why are those two not the same?

Thanks for your help!!
KR
Leonard

Answer 1

I believe the issue is coming in because you are representing events as rates where you have 0 events.

Let's walk through how the offset is added (following the notation from the linked answer):

A poisson regression of the form: $log(\mu_x) = \beta_0 + \beta_1x$ is adjusted for different measurement times by correcting the expected counts ($\mu_x$) for the exposure time ($t_x$): $$log({\mu_x\over t_x}) = \beta_0 + \beta_1x$$

This is then simplified to introducing the offset by the rules of log transformations to give you: $$log(\mu_x) = log(t_x) + \beta'_0 + \beta'_1x$$

Note that the parameter $\mu_x$ can never be 0 because otherwise it's log is undefined (log(0) is forbidden).

So now let's take a look at what happens when you just calculate the rates of your model when you have 0 counts. For example take the first few rows of your data table: rows 1-3 have nb ~ 0.25 and rows 4-6 have nb ~ 0.47 but there are no counts for any of them. If you ratio these however all six rows have the same value all of a sudden: 0.

So the issue then is: zero is a difficult number to deal with because division and multiplication doesn't change the value.

So when you transformed your data by taking the ratio you actually slightly changed the data set that was being fit and that's why you have different answers.