3
$\begingroup$

I am building GAMs of the following form (using R's mgcv notation) and would like to compare the fit between each model.

model1 <- gam(dependent_var ~ metric +
                             s(latitude,longitude) + 
                             s(UNIT, bs = 're'),
                             data  = df)
model2 <- gam(dependent_var ~ metric2 +
                             s(latitude,longitude) + 
                             s(UNIT, bs = 're'),
                             data  = df)

I would normally think to use AIC or a LRT to compare fit, but note that the difference between the models is only metric and metric2, i.e., the models are not truncated versions of one-another.

Another option I have considered is to build training and testing datasets, and compare the sum of the residuals in the testing dataset to compare fit. Is there a proper way to compare fit using two, non-truncated GAMs?

$\endgroup$

1 Answer 1

2
$\begingroup$

Answer

For GAMs, one does not need nested models for AIC/BIC comparison. Simon Wood even notes this in his canonical book on GAMS (p.110) and later discusses this in much greater detail in p.301.

Models under comparison need not be nested, although some neglected terms in the approximation of the expected K-L divergence will cancel if they are.

Reference

Wood, S. N. (2017). Generalized additive models: An introduction with R (2nd ed.). CRC Press, Taylor and Francis Group.

$\endgroup$
3
  • $\begingroup$ To confirm, AIC is still useful under the same rules (smaller = better fit) when using non nested GAMs? $\endgroup$ Commented Mar 8 at 15:18
  • 2
    $\begingroup$ It's not just for GAMs, AIC in general does not require nested models. $\endgroup$ Commented Mar 8 at 15:38
  • $\begingroup$ @geoscience123 yes, lower scores are preferred and as Gavin noted this applies across contexts. $\endgroup$ Commented Mar 8 at 20:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.