0
$\begingroup$

I fitted a generalized additive model (GAM) using SAS PROC GAMPL with a Poisson distribution and a log link function.

The outcome was a measure of repair cost for a large collection of units (4k+). There is a percentage of units with values that are zero ($0) which I expected to simply drop out of the model estimation process due to the log transformation. However, the reported totals for the used observations do not reflect this.

Are these observations really being used? If so, does anyone know how?

Thanks

$\endgroup$
2
  • $\begingroup$ What do you mean by “zero values”? The response variable or a weight? $\endgroup$ Commented Mar 6, 2022 at 22:13
  • $\begingroup$ The response value is zero. I've edited to try and make things as clear as possible. The question is about estimation for GAM with a log link function and how it handles Y=0. $\endgroup$
    – Vu Xing
    Commented Mar 6, 2022 at 22:33

1 Answer 1

2
$\begingroup$

If you choose a GAM with a Poisson distribution and log-link, it means that

  • the response is modelled by a Poisson random variable $Y$,
  • the mean $E[Y]$ is related to the nonlinear predictor $\sum f(x_i)$ via the link function, i. e. $$log\left(E[Y]\right) = x_0 + \sum f(x_i) \Leftrightarrow E[Y] = exp\left(x_0 + \sum f(x_i) \right)$$

A Poisson random variable can be zero with positive probability, i. e. there is no reason to exclude these observations. In that sense, $y=0$ is not different from $y=1$ or any other value. There is no log transformation of the response variable.

Side note: If there are weights for each observation (if you don't specify them, they are equal to $1$ for all observations), these weights would be log-transformed into an offset $$E[Y] = W \cdot exp\left(x_0 + \sum f(x_i) \right) = exp\left(log(W) + x_0 + \sum f(x_i) \right). $$ Here, observations with weight zero have to be excluded. Weights can be used when trying to model ratios. An example is modelling the claims frequency in insurance, where $Y$ would be the number of claims and $W$ the number of insurance contracts.

$\endgroup$
4
  • $\begingroup$ Your discussion of weights is mysterious. What role do you have in mind for them and why would weights be transformed when transforming a variable? $\endgroup$
    – whuber
    Commented Mar 7, 2022 at 3:34
  • $\begingroup$ I added an explanation for considering the weights. They came to mind, when thinking about log transforming in this context. $\endgroup$ Commented Mar 7, 2022 at 8:22
  • $\begingroup$ Thank you: that's an interesting and useful observation. +1 $\endgroup$
    – whuber
    Commented Mar 7, 2022 at 13:23
  • $\begingroup$ Thank you for your thorough answer, weights included. $\endgroup$
    – Vu Xing
    Commented Mar 8, 2022 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.