GLM: verifying a choice of distribution and link function

Question

I have a generalized linear model that adopts a Gaussian distribution and log link function. After fitting the model, I check the residuals: QQ plot, residuals vs predicted values, histogram of residuals (acknowledging that due caution is needed). Everything looks good. This seems to suggest (to me) that the choice of a Gaussian distribution was quite reasonable. Or, at least, that the residuals are consistent with the distribution I used in my model.

Q1: Would it be going too far to state that it validates my choice of distribution?

I chose a log link function because my response variable is always positive, but I'd like some sort of confirmation that it was a good choice.

Q2: Are there any tests, like checking the residuals for the choice of distribution, that can support my choice of link function? (Choosing a link function seems a bit arbitrary to me, as the only guidelines I can find are quite vague and hand-wavey, presumably for good reason.)

Answer 1

1. This is a variant of the frequently asked question regarding whether you can assert the null hypothesis. In your case, the null would be that the residuals are Gaussian, and visual inspection of your plots (qq-plots, histograms, etc.) constitutes the 'test'. (For a general overview of the issue of asserting the null, it may help to read my answer here: Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?) In your specific case, you can say that the plots show your residuals are consistent with your assumption of normality, but they don't "validate" the assumption.

2. You can fit your model using different link functions and compare them, but there isn't a test of a single link function in isolation (this is evidently incorrect, see @Glen_b's answer). In my answer to Difference between logit and probit models (which may be worth reading, although it isn't quite the same), I argue that link functions should be chosen based on:

   1. Knowledge of the response distribution,
   2. Theoretical considerations, and
   3. Empirical fit to the data.

   Within that framework, the canonical link for a Gaussian model would be the identity link. In this case you rejected that possibility, presumably for theoretical reasons. I suspect your thinking was that $Y$ cannot take negative values (note that 'does not happen to' is not the same thing). If so, the log is a reasonable choice a-priori, but it doesn't just prevent $Y$ from becoming negative, it also induces a specific shape to the curvilinear relationship. A standard plot of residuals vs. fitted values (perhaps with a loess fit overlaid) will help you identify if the intrinsic curvature in your data is a reasonable match for the specific curvature imposed by the log link. As I mentioned, you can also try whatever other transformation meets your theoretical criteria that you want and compare the two fits directly.

Answer 2

Would it be going too far to state that it validates my choice of distribution?

It kind of depends on what you mean by 'validate' exactly, but I'd say 'yes, that goes too far' in the same way that you can't really say "the null is shown to be true", (especially with point nulls, but in at least some sense more generally). You can only really say "well, we don't have strong evidence that it's wrong". But in any case we don't expect our models to be perfect, they're models. What matters, as Box & Draper said, is "how wrong do they have to be to not be useful?"

Either of these two prior sentences:

This seems to suggest (to me) that the choice of a Gaussian distribution was quite reasonable. Or, at least, that the residuals are consistent with the distribution I used in my model.

much more accurately describe what your diagnostics indicate -- not that a Gaussian model with log link was right -- but that it was reasonable, or consistent with the data.

I chose a log link function because my response variable is always positive, but I'd like some sort of confirmation that it was a good choice.

If you know it must be positive then its mean must be positive. It's sensible to choose a model that's at least consistent with that. I don't know if it's a good choice (there could well be much better choices), but it's a reasonable thing to do; it could well be my starting point. [However, if the variable itself is necessarily positive, my first thought would tend to be Gamma with log-link, rather than Gaussian. "Necessarily positive" does suggest both skewness and variance that changes with the mean.]

Q2: Are there any tests, like checking the residuals for the choice of distribution, that can support my choice of link function?

It sounds like you don't mean 'test' as in "formal hypothesis test" but rather as 'diagnostic check'.

In either case, the answer is, yes, there are.

One formal hypothesis test is Pregibon's Goodness of link test[1].

This is based on embedding the link function in a Box-Cox family in order to do a hypothesis test of the Box-Cox parameter.

See also the brief discussion of Pregibon's test in Breslow (1996)[2] (see p 14).

However, I'd strongly advise sticking with the diagnostic route. If you want to check a link function, you're basically asserting that on the link-scale, $\eta=g(\mu)$ is linear in the $x$'s that are in the model, so one basic assessment might look at a plot of residuals against the predictors. For example,

working residuals $r^W_i=(y_i-\hat{\mu}_i)\left(\frac{\partial \eta}{\partial\mu}\right)$

(which I'd lean toward for this assessment), or perhaps by looking at deviations from linearity in partial residuals, with one plot for each predictor (see for example, Hardin and Hilbe, Generalized linear models and extensions, 2nd ed. sec 4.5.4 p54, for the definition),

$\quad r^T_{ki}=(y_i-\hat{\mu}_i)\left(\frac{\partial \eta}{\partial\mu}\right)+x_{ik}\hat{\beta}_k$

$\qquad\:=r^W_i+x_{ik}\hat{\beta}_k$

In cases where the data admit transformation by the link function, you could look for linearity in the same fashion as with linear regression (though you my have left skewness and possibly heteroskedasticity).

In the case of categorical predictors the choice of link function is more a matter of convenience or interpretability, the fit should be the same (so no need to assess for them).

You could also base a diagnostic off Pregibon's approach.

These don't form an exhaustive list; you can find other diagnostics discussed.

[That said, I agree with gung's assessment that the choice of link function should initially be based on things like theoretical considerations, where possible.]

See also some of the discussion in this post, which is at least partly relevant.

[1]: Pregibon, D. (1980),
"Goodness of Link Tests for Generalized Linear Models,"
Journal of the Royal Statistical Society. Series C (Applied Statistics),
Vol. 29, No. 1, pp. 15-23.

[2]: Breslow N. E. (1996),
"Generalized linear models: Checking assumptions and strengthening conclusions,"
Statistica Applicata 8, 23-41.
pdf