1
$\begingroup$

I am using GAM models with identity link function and Gaussian family of distributions, more specifically the mgcv package in R. I observed that in this setting, the raw residuals (prediction - real value) are identical to the deviance residuals and the Pearson residuals, and normally distributed. In my understanding, the GAM models assume that the response variable in each observation is drawn from a potentially different distribution within an exponential family, which in my case is Gaussian, and approximate the means of these distributions. I might be confused here, but I don't recall any strong assumptions on the variances of these Gaussians, therefore I don't understand why are the raw residuals equal to the Person or deviance residuals. I imagine the situation in the following way:

The response variable in each observation is drawn from a different Gaussian, with a potentially different variance. I fit a GAM model to the expectations of these Gaussians and compute the raw residuals. For any given observation, the corresponding raw residual tells me how much my prediction is off from the observed value, but that distance has two components. One comes from the error of the model's fit to the expected value for this observation, and the other component comes from the observed value being sampled from this Gaussian distribution instead of simply being equal to the mean. Now, since the variances of the Gaussians for the different observations can be different, it is possible that in some "direction" in the data, this variance is growing. This would lead to a systematic growth in the residuals that is not coming from a lower quality of the fit of the GAM model. If I understand correctly, the normalization in the Person residuals, as well as the deviance residuals, are taking care of such phenomena. In particular, I don't see a reason for any of these residuals to be normally distributed unless stronger assumptions on the shape of the Gaussians is implied. I have been reading Simon Wood's book and I know that some mean-variance relationships are in fact implied, but I feel very confused. Any help with an explanation or a reference would be much appreciated.

$\endgroup$
3
$\begingroup$

A Gaussian GAM with identity link is just an ordinary linear model. For the Gaussian response family, there is no mean-variance relationship (the variance doesn't depend on the mean) and the three types of GLM residuals all coincide. Any book on GLM theory will tell you this.

When you fit a GAM or GLM, you choose the response family according to what mean-variance relationship you expect the data to have. You have chosen the Gaussian family, which corresponds to constant variance.

The response variable in each observation is drawn from a different Gaussian, with a potentially different variance.

No, the Gaussian model assumes all the variances to be the same. Each observation has potentially a different mean, but the variance is constant.

it is possible that in some "direction" in the data, this variance is growing.

Not for the Gaussian family.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Isn’t it an additive model, not linear? $\endgroup$ – Dave Jun 27 at 11:01
1
$\begingroup$

I found the answer in Simon Wood's book on GAM models, in the chapter about GLM, just above formula $(3.2)$.

A distribution is said to belong to the exponential family of distributions if its probability density function can be written as:

$ f_\theta(y) = \exp \left( \frac{y \theta - b(\theta)}{a(\phi) + c(y,\phi)} \right) , $

where $b, a$ and $c$ are arbitrary functions, $\phi$ is scale parameter, and $\theta$ is the canonical parameter of the distribution. Now that we have the context, I quote the book (just before Chapter 3.1.2):

" However, when $\phi$ is unknown matters become awkward, unless we can write $a(\phi) = \frac{\phi}{\omega}$", where $\omega$ is a known constant. This restricted form in fact covers all the cases of practical interest here. For example, $a(\phi) = \frac{\phi}{\omega}$ allows the possibility of unequal variances in the models based on the normal distribution, but in most cases $\omega$ is simply $1$."

So, both GLM and GAM models permit, in theory, for a Gaussian response variable to have varying variance across observations. However, for some reason, this seems not to be used in practice and equal variance is assumed instead. In the case of a GLM, this means we end up with a linear model and the equivalence of the different types of residuals is clear. For a GAM model, we end up with a penalized linear model, so one (I) would need to work out the formuli, but intuitively it is clear that the residuals should coincide.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.