I know similar questions have come up a lot but I'm still confused on how to model interactions in GAM (using mgcv in R). In my analysis, my response variable has normally distributed residuals and the variable is related to three continuous variables.

My goal is to predict values of y over a range of values of the continuous predictors. Also, I would like to compare the estimated slopes (and smooths?) to simulated data. I believe there are interactions between the continuous predictors.

Upon visual inspection it seems the relationship between y and one of the continuous predictors is non-linear. Hence, I will use GAM. With two linear predictors and one non-linear predictor what would be the appropriate model? One that simply includes all interactions in a tensor product?

y = a + te(x1, x2, x3)

But if x2 and x3 are linearly related to y then:

y = a + te(x1, x2) + te(x1, x3) + b1(x2) + b2(x3) + b3(x2:x3)


y = a + s(x1) + b1(x2) + b2(x3) + b3(x1:x2) + b4(x1:x3) + b5(x2:x3)

Or, we can throw ti() into the mix:

y = a + ti(x1) + b1(x2) + b2(x3) + ti(x1:x2) + ti(x1:x3) + b3(x2:x3)


2 Answers 2


Your question seems to confuse a few separate issues. First forget about functional forms and assume that linear models are appropriate. You've got three variables that potentially predict $y$. You should interact them if the level of one mediates the influence of the others. So, lets say that $y$ is the happiness of cats, $x_1$ is cat food quantity, and $x_2$ is the distance of the cat from the nearest cat food, then you might imagine that the effect of $x_1$ on $y$ is mediated by $x_2$.

Now, assume that cats have declining marginal utility. You want to model s(x1) to get the shape of that curve. But you suspect that it might only work for nearby food, and that far-off food makes a cat less happy (because they have to get up to go get it). Now, if the effect of distance in mediating the happiness effect of quantity changes the same amount over all distance increments, you're good. y~s(x1,by=x2). Linear interaction. If on the other hand the effect of distance in mediating the happiness effect of quantity is non-linear (maybe the cat doesn't mind walking a 10 steps, and doesn't mind walking 20, and is indifferent between 10, 20, and 40, but can't be made to walk 1000 steps for any quantity of cat food), then you need a nonlinear interaction. y~te(x1,x2).

You should apply the same sort of logic to 3-way interactions. Maybe this entire relationship is mediated by the phase of the moon. If cats have linear utility and enjoy long walks when the moon is full, then y~te(x1,x2,by=FULLMOON), where FULLMOON is a dummy. If however there is a continuous monthly cycle in which cats change from insatiable walkers to being contented and lazy, you'd need something like y~te(x1,x2,MOONPHASE). Where moonphase is continuous.

Edit: For model selection between reasonable candidate models underuncertainty about which one makes the most sense, try cross validation http://en.wikipedia.org/wiki/Cross-validation_%28statistics%29

  • $\begingroup$ Thank you Anna and ACD for your answers. Wrongly, I thought that "by" worked only with factors. Two follow-up questions: 1) if I modeled y~s(x1, by=x2) would I still add "main effects," i.e. s(x1) and x2? Anna's answer implies "yes" but ACD's answer implies "no." I know Simon Wood responded to a similar question on r help in 2011 but I didn't quite understand the answer. 2) if I decided that x2 should also be smoothed (s(x2)) then would that invalidate the "by" meaning that the interaction would necessarily have to be modeled as te(x1, x2)? $\endgroup$
    – r_e_f
    Commented Feb 17, 2014 at 14:17
  • $\begingroup$ Any model with $y$, $x_1$ and $x_2$ will give you a conditional expectation of $y|X_1=x_2,X_2=x_2$. If you swap the values of x1 and x2 in a model with a tensor interaction, you don't get the same answer unless the interaction is totally smooth. In a linear model, the two are equivalent. Note that I've never read this anywhere authoritative as a justification for omitting main effects in tensors, but it represents my parsing of that inscrutable Simon Wood advice that you reference. If wrong somehow, please let me know. In the by case, you want main effects unless you're sure they =0 $\endgroup$ Commented Feb 17, 2014 at 22:05
  • $\begingroup$ Re: the second part of your comment, s(x1,by=x2) implies that the effect of x1 is nonlinear, but that it is linearly scaled by x2. It is hard to imagine a situation where such would hold AND x2 would reasonably be expected to be nonlinear. Are you trying to avoid tensors to keep degrees of freedom down? If so, I know the feeling... $\endgroup$ Commented Feb 17, 2014 at 22:24

Suppose that x1 is not linear and x2 and x3 are and they interact between both with x1, then: y=a+s(x1) + s(x1,by=x2)+s(x1,by=x3)

otherwise if x2 and x3 interact between them, so it will be: y=a+s(x1) + x1+x2+x1*x2

if all interact then: y=a+s(x1)+te(x1,x2,x3)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.