In mgcv::gam(), what is the difference between s(x, by=cat) and s(x, cat, bs='fs')? In this comment about smooth-factor interactions, Gavin Simpson wrote:

s(x1, by = cat) doesn't create a "fs" class smooth

I understand that
y ~ s(x, by = cat) + cat             (a)
//cat is an unordered factor

creates a separate smooth for each level of the factor as shown in summary() and plot() with a confidence interval around each smooth (and that this model doesn't say anything about whether the smooths for those levels are different).
On the other hand, in
y ~ s(x, cat, bs = 'fs')             (b)

the summary shows one smooth, but the plot() shows two smooths and with no confidence intervals.
So my questions are:

*

*How should the fs smooth in (b) summary() be interpreted? Does the p-value say anything about whether the smooths for cat levels are different?


*Why does plot() in (b) shows two curves without confidence intervals?


*What is the exact difference between (a) and (b)? When should I use one instead of the other? And how does their interpretation differ?
Many thanks
 A: Q3
The by smooth variant has a separate smoothness parameter for each level of cat. In one sense you can think of these smooths as creating entirely separate smooth functions (they're aren't quite, they share the same basis for example), hence you have nlevels(cat) entries in the summary() output.
fs smooths share a single smoothness parameter, and in that sense are a bit closer to the smooth equivalent of random slopes; all the smooths are being pulled towards 0 (null function), with the smooths for some levels of cat pulled more towards 0 than others.
Q1
You should just treat the fs smooth summary as some measure of overall importance of the set of smooth functions. If the fs smooth is the only one in the model where x is a covariate then you could say interpret this as a test of the amount of variation in the response that can be explained by smooth functions of x, against a null of no effect. It's like an omnibus test in the sense that it is over the set of smooths.
Put another way, it is a bit like a test for a variance term of random slopes in a mixed effects model; this would be on a single variance parameter and you get a single test over the entire "effect" even though it represents $m$ slopes.
Q2
Typically, one would use fs where there where nlevels(cat) was large and you aren't particularly interested in the specific levels, just like you might work with random effects in a mixed effects model.
As there could feasibly be a separate line for the many levels of the factor but we think of this as a single "smooth" you get a single plot with each smooth superimposed to get some idea of the variation within and between individual functions. It would get very messy it if also showed confidence bands for each smooth.
In the by case, because you conditioned on the observed set of levels (sensu fixed effects in a mixed model) it makes sense to view these are being of individual interest, hence a separate plot per level of cat.
Also note that this is just the default plot to get a quick look at the fitted functions.
