Based on my readings about GLM, I am pretty sure that when we say the distribution of the response variable $y$ is a member of exponential family of distribution, what we really mean is that conditioning on $x$, $y$ is a member of the exponential family distribution.

For example, say $y \sim \mathrm{Poisson}(\lambda)$. What we really mean is, given $x$, $\lambda$ is fixed through link function $g(\lambda)=\beta x$, and $y$ is a Poisson distributed with mean $\lambda$. So $y$ itself might not be a member of exponential family of distribution (unless it has been proved that collection of exponential family of distribution with different means is also exponential family of distribution).

And the distribution of $y$ depends on (the vector) $x$.

Now my comments/questions are:

  1. We should not look at the distribution of $y$ to determine the error structure. What we should do really is to look at the conditional distribution of $y$ to determine the error structure.

  2. How does this affect our diagnostics, e.g. residual plots — i.e. we should really not expect the (unconditional) residuals to be have a random pattern around the origin?

  1. You're correct on this point. This is why it doesn't make sense to try to assess the suitability of (say) a Poisson model for $Y$ by looking at $Y$ alone -- it might look distinctly non-Poisson because it's effectively a mixture of Poissons, with the mixture determined by the pattern of the predictors and the population parameters.

  2. Residuals are conditional (if the model is correct).

    That is, the $i$th residual is an estimate of the "error" $Y_i-E(Y_i|\mathbf{X_i}=\mathbf{x_i})$

    As a result, pattern in the distribution about zero (consistent smooth deviations for example) indicates that the model for the mean is not correct; some other indications of model inadequacy can also be seen in a plot of residuals (e.g. against fitted or against specific predictors).

In GLMs, however, it's generally more useful to think of the model not as a "mean + error" model (because the distribution of the error is different for different observations) but directly in terms of the conditional distribution. We still use (various kinds of) residuals for model assessment (diagnostics), of course, but in order to inform us about specification errors in the conditional distribution of $Y$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.