# GAMM using mgcv in R: by participant random effects for categorical variables

I am fitting a GAMM using mgcv. I have two variables, one a categorical 2-level variable difference, the other a continuous variable Learning_stage. I would like by participant random effects for difference, specificly random slopes and intercepts. A stackoverflow link had an answer saying that factor variables with bs='fs' result in intercepts for every factor level. A smooth function with bs='re' should give random slopes if I am correct.

Would that mean I would need to add the terms s(difference, participant, bs='re') as well as s(difference, participant, bs=’fs’, m=1)?

The model defined so far:

model <- bam(error ~ difference + s(Learning_stage) + s(Learning_stage, by=difference) + s(Learning_stage, participant, bs=’fs’, m=1), data=data_for_gamm)


You want

model <- bam(error ~ s(participant, bs = "re") +
s(participant, difference, bs = "re"), data = data_for_gamm)


If you want participant random effects and random "slopes" of difference per participant, as per your stated requirement.

The "fs" smooths does much more than add random intercepts; it represents a fully penalized spline basis where all smooths share the one smoothing parameter. As it is fully penalized, s(x, f, bs ="fs") has a penalty for the constant term in the basis (which means we have a random intercept for each level of the f), another penalty for the linear term in the basis (se we have random slopes of x per level of f), and then a random smooth function of x per level of f, noting that the x in these smooths must be continuous.

The "re" basis only gives a random "slope" if you add a grouping variable and the variable you want random slopes for. In your case you have two factors (participant and difference) but this still works, you'll just get a different estimate of difference for each level of participant.

If you want a smooth of Learning_stage, you have to state what you want that smooth to do. Right now you have a model that asks for:

1. ~ difference fixed effect for levels of difference,
2.  + s(Learning_rate) an average smooth effect of Learning_rate (assuming difference is a normal factor? If it is an ordered factor, things are different),
3. + s(Learning_rate, by = difference) a separate smooth effect of Learning_rate for each level of difference. This smooth almost certainly needs m=1 adding to it to make it more easily identifiable from the "average" smooth effect (2.), and
4. + s(Learning_rate, participant, bs = "fs") a random smooth effect of Learning_rate per participant, with smooths sharing the one wiggliness parameter for the smooths. This term also includes random intercepts per participant and a random slope of Learning_rate as constant and linear terms are in this basis and are fully penalized. You likely don't need m=1 here.

This is a lot of smooth effects of Learning_rate; do you need all of them? There's nothing wrong with the model as you have it, but it is quite a complex starting model.

• Thank you for reply. Difference is a normal factor indeed. I should have been clearer that I would like to ADD the random effects, not just have the random effects, my bad. I am used to "full" models with lme4. I also need the model to "answer" quite some questions, hence the complexity. Dec 5, 2023 at 22:18
• Does the order in s(participant, difference, bs='re') matter? Could it also be s(difference, participant, bs='re')? Dec 5, 2023 at 22:20