I have speech signals sampled in 10-ms intervals ('$time$') in 8 different geographical regions ('$region$'), from 20 subjects each. For each of these regions, I want to know if the sampled trajectory over time is different from the grand mean.

In linear regression, I would create a sum contrast ($-0.5$ for an arbitrary reference region, seven columns of $+0.5$ for the others) and fit the below model (leaving out random effects for simplicity and ignoring the obvious error term):

$ y \sim \beta_0 + \beta X_{time} + \beta X_{contrast[1..7]} + \beta X_{time \times contrast[1..7]} $

However, I have reason to believe that the effect of $time$ is nonlinear and hence want to fit a generalized additive model. I could fit the below model:

$ y \sim \beta_0 + \beta f(X_{time}) + \beta X_{contrast[1..7]} + \beta f(X_{time \times contrast[1..7]}) $

with $f$ a smoothing function such as a cubic regression spline. However, I could also fit the following model:

$ y \sim \beta_0 + \beta f(X_{time}) + \beta X_{region[1..8]} + \beta f(X_{time \times region[1..8]}) $

where $region[1..8]$ is eight indicator columns. The linear regression equivalent of this approach

$ y \sim \beta_0 + \beta X_{time} + \beta X_{region[1..8]} + \beta X_{time \times region[1..8]} $

is obviously not identifiable, but I am unsure whether the same is true for the GAM approach. The reason for my uncertainty is that smooths are penalized, so I would expect that if one of the terms is not identifiable, it will just be penalized out of the model (i.e. estimated as 0). In fact, I can fit the GAM model just fine, using R package mgcv:

m <- gam(y ~ region + s(time,bs='cr') + s(time,bs='cr',by=region))

and the results seem to be in line with my assumption: some of the smooths are estimated as straight lines, presumably due to lack of support from the data due to identifiability problems, but all in all the results look somewhat credible.

Is there an equivalence relationship between the identifiability of the linear regression model and a similarly-parametrized GAM, or is it fine to fit this overparameterized GAM?


The GAM in question is subject to identifiability constraints, which may be helping here. However, you will need to take care with models of this type, which is a global smooth plus difference smooths.

One thing that can help is to put a different penalty on the difference smooths; penalising based on the first derivative of the smooth is natural here as the region specific smooth involves a contribution from the main s(time) term plus some smooth deviation from this global smooth.

To choose a penalty based on the first derivative of the smooth, use m = 1 in the s() call

m <- gam(y ~ region + s(time,bs='cr') + s(time,bs='cr',by=region, m = 1))

This penalty, only on the smooth deviation term, penalises any departure from a flat function, which when added to the global time smooth is really any departure from the global smooth.

  • $\begingroup$ You seem to agree with my initial thoughts that the identifiability constraints, compounded with your suggestion of a more severe penalization for the difference smooths (which I like as an idea), make this feasible. I recently stumbled on an alternative option, coincidentally on your blog: fromthebottomoftheheap.net/2017/10/10/difference-splines-i. Here you omit the reference smooth and compare the by=... conditions post-hoc using the lpmatrix (in my case, I could for each comparision average the lpmatrices from the 7 remaining conditions). Would that perhaps be even better? $\endgroup$ Jul 30 '18 at 9:48
  • $\begingroup$ @LostLinguist There are quite a few ways you can specify a model for essentially different splines per level of a grouping factor, two of which you've now come across. Which to choose depends on whether you need the global of common smooth, whether you want smooths to have different smoothness parameters or the all share the same. I and some colleagues are writing a paper (with an ecological focus) that looks at many of the options with mgcv: github.com/noamross/mixed-effect-gams . The manuscript is reasonably complete with just a few details to iron out. $\endgroup$ Aug 1 '18 at 21:57
  • $\begingroup$ Thanks, that's a wonderful paper to have as a resource! Accepting answer and I'll think in more detail about which of these methods suit my data best. $\endgroup$ Aug 3 '18 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.