How I can interpret GAM results?

Question

I have a question about Generalized Additive Models. What is deviance explained, GCV score and scale est in GAM results? What do these indicators show?

Answer 1

The deviance explained is a bit like $R^2$ for models where sums of squares doesn't make much sense as a measure of discrepancy between the observations and the fitted values. In generalised models instead we measure this discrepancy using deviance. It is computed using the likelihood of the model and hence has a somewhat different mathematical definition for each error distribution (family argument in glm()/gam()). In the case of Gaussian models estimated as a GLM/GAM, deviance and residual sums of squares are equivalent.

The deviance $D$ of a model is defined as:

$$ D = 2 \left [ l(\hat{\beta}_{\mathrm{max}}) - l(\hat{\beta}) \right ] \phi $$

where $l(\hat{\beta}_{\mathrm{max}})$ is the maximised likelihood of the saturated model and $l(\hat{\beta})$ is the maximised likelihood of the model you've fitted. The saturated model is a model with one parameter for each data point; you can't get a higher likelihood than this, given the data. $\phi$ is the scale parameter. The scaled deviance is simply

$$D^{*} = D / \phi$$

These scaled deviances play a role in likelihood ratio tests, where the difference of scaled deviances for two models is $\sim \chi^2_{p_1, p_2}$ (chi-square distributed with degrees of freedom $p_1$ and $p_2$).

Deviance explained is just representing the above as the proportion of the total deviance explained by the current model.

The scale estimate is $\hat{\phi}$, i.e. this is the value of $\phi$ estimated during model fitting. For the Poisson and Binomial families/distributions, by definition $\phi = 1$, but for other distributions this is not the case, including the Gaussian. In the Gaussian case, $\hat\phi$ is the residual standard error squared.

The GCV score is the minimised generalised cross-validation (GCV) score of the GAM fitted. GCV is used for smoothness selection in the mgcv package for R; smoothing parameters are chosen to minimise prediction error where $\phi$ is unknown, and standard CV or GCV can be used to estimate prediction error. GCv is preferred here as it can be calculated without actually cross-validating (refitting the model to subsets of the data) it, which saves computational time/effort. The value reported is the minimised GCV score (UBRE, Un-biased Risk Estimator, scores are shown instead you are fitting a model with known $\phi$), and you can use these scores a bit like AIC, smaller values indicated better fitting models.

GAMs fitted using GCV smoothness selection can suffer from under-smoothing. This can happen where the GCV profile is relative flat and random variation can lead to the algorithm converging at too wiggly a fit. Fitting via REML (use method = "REML" in the gam() call) or ML has been shown by Simon Wood and colleagues to be much more robust to under smoothing, but at computational expense.

The above summaries are based on the descriptions in Simon Wood's rather excellent book on GAMs:

Wood, S. N. (2006). Generalized Additive Models: An Introduction with R. Chapman and Hall/CRC.