6
$\begingroup$

I am fitting GAMs (in R using mgcv) to count data and am uncertain about how to select from among competing models. In my model specification I am using "ts" as my smoothing basis and a negative binomial distribution. As I understand it this essentially adds an extra penalization to the the smooth and thereby has the potential to help exclude some terms completely during the model fitting.

The modeling strategy I employed is a follows

I first fit a global model (i.e. with all my independent variables of interest) e.g., (N~s(x1)+s(x2)+s(x3)+s(x4)). When I find that the term x3 has an EDF that is practically equal to 0 and that x4 is not significant at the 0.05 level (but with EDF>1) I subsequently fitted a model by eliminating these two terms i.e. N~s(x1)+s(x2).

What does it mean when I compare the the two models and find that the nested model (now with all terms significant and non zero EDFs) has a lower deviance explained? Also is this a defensible approach to identify important predictor variables?

Any advice will be greatly appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
9
$\begingroup$

If you are using an extra penalty on each term, you can just fit the model and you are done (from the point of view of selection). The point of these penalties is allow for shrinkage of the perfectly smooth functions in the spline basis expansion as well as the wiggly functions. The results of the model fit account for the selection/shrinkage. If you remove the insignificant terms and then refit, the inference results (say in summary() output) would not include the "effect" of the previous selection.

Assuming you have a well-chosen set of covariates and can fit the full model (a model with a smooth of each covariate plus any interactions you want) you should probably just work with the resulting fit of the shrunken full model.

If a term is using effectively 0 degrees of freedom it is having no effect on the fit/predictions at all. For the non-significant terms that have positive EDFs, by keeping them in you are effectively stating that these covariates have a small but non-zero effect. If you remove these terms as you suggest, you are saying explicitly that the effect is zero.

In short, don't fit the reduced model; work with the full model to which shrinkage was applied.

The deviance explained of the reduced model can be lower as it has fewer terms with which to explain variation in the response. It's a bit like the $R^2$ of a model increasing as you add covariates.

Improve this answer
$\endgroup$
4
  • $\begingroup$ well explained. Up to now I have followed the same logic. But I noticed that some authors (Wood, 2006, for example) fit a model with shrinkage, and if some terms have 0 degrees of freedom, they re-fit a model without those terms, and using it for inference. Any thoughts on that? $\endgroup$
    – Tilen
    Commented Mar 1, 2017 at 0:02
  • $\begingroup$ @Tilen there is precedence for that; in the other shrinkage world, there is the relaxed lasso which involves selection via the lasso shrinkage penalty then refitting with only those variable with non-zero effect to get OLS estimates. I will add that that approach like the lasso is often see as a prediction tool; until recently it wasn't possible to do inference on a lasso model as there was no supporting theory. From the point of view of inference, I would leave those zero-effect variables in the GAM & take the degrees of freedom hit which accounts for the selection. $\endgroup$
    – Gavin Simpson
    Commented Mar 1, 2017 at 0:21
  • 2
    $\begingroup$ @Tilen also note that Simon Wood's book came out about 6 years before the paper by Marra and Wood which describes the select = TRUE approach. $\endgroup$
    – Gavin Simpson
    Commented Mar 1, 2017 at 0:23
  • $\begingroup$ cool, that's really useful feedback - thanks! $\endgroup$
    – Tilen
    Commented Mar 1, 2017 at 18:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.