AIC comparison of GAMM and LMM: Is it valid?

Question

I have 2 models, using exactly the same variables, one fitted as a linear mixed model (LMM) and another fitted as a generalised additive mixed model (GAMM). I am interested in the fixed effect X and whether it is more appropriate (or just equivalent) to fit as a linear term in a simpler linear model. As I understand, restricted maximum likelihood is not used to compare fixed effects, so I include REML = FALSE. Is a comparison of these two models with AIC valid? I am not certain because of how the smooth term is treated in the GAMM and whether it is the same as a normal fixed effect?

library(lmerTest)
m1 <- lmer(Y ~ X + W + (1|V) + (1|U), REML = FALSE)

library(gamm4)
m2 <- gamm4(Y ~ s(X) + W + (1|V) + (1|U), REML = FALSE)

(Where: Y = continuous response, X = continuous explanatory variable, W = continuous covariate, V = factor block-like variable, U = individual variable (there are repeated measures from individiuals))

Answer 1

You have to be very careful with off-the-shelf AIC calculations for mixed models and for GAMs. For the latter, you want the AIC to account for having done smoothness parameter selection for example.

There is a clean way to do the test you want however:

m <- gamm4(Y ~ X + s(X, m = c(2,0)) + W + (1|V) + (1|U), REML = TRUE)

In this model, we include the linear term explicitly as a parametric effect and then we create a smooth of X that only includes wiggly bits - the m = c(2, 0) means we want the usual 2nd derivative penalty on the smooth but using a basis with no null space. The null space is the span of functions to which penalty doesn't apply. The functions in this null space are the constant and linear terms. This way the smooth represents only wiggly/smooth effects of X on Y, while the linear effect of X on Y is modelled using the parametric term.

You can then use the test in the summary(m$gam) to statistically decide if the non linearity is required.