Fitting lower-order interactions with higher order main effects

Question

We have 2 continuous predictors in a regression model, and both have significant quadratic terms. Our main hypothesis we'd like to test is (generally) whether there is an interaction between these variables. When modelling, we see a significant interaction between the linear terms. But interactions terms involving the quadratic terms are not signficant.

So are there any problems with fitting the model with the quadratic main effects and only first order interaction terms (e.g. $y = intercept + x^2 + x + xz + z + z^2$)? I don't know of any theoretical problems with this, but looking at the predicted outcome, this seems to force a strange relationship -- specifically, looking at curves of $y$ versus $x$, for all values of $z$ the curves intersect at a specific point.

In short, the question is, if you are set on including a quadratic covariate, are you then only interested in testing interactions of the highest ordered terms (and if only linear interactions are significant then this means there is no interaction.

Answer 1

Firstly, it's perfectly sane to fit the model given by the following parameterization (I'll call this the "first approach"):

$$ E[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 + \gamma X W $$

and you can use the 1 dimensional Wald test for $\gamma = 0$.

However, the "full model" you described is peculiar $$ E[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 + \gamma_1 X W + \gamma_2 X^2W^2 $$

when, as I read it, you should have instead fit the following model (we'll call this the "second approach"):

$$ E_{full}[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 + \gamma_1 X W + \gamma_2 XW^2 + \gamma_3 X^2W + \gamma_4 X^2 W^2 $$

tested against the non-interaction effect models:

$$ E_{null}[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 $$

your nested model test is the joint test of $\gamma_1, \gamma_2, \gamma_3, \gamma_4 =0 $. If you reject this null hypothesis, you conclude there is an interaction. But 4 terms is complex.

As long as the interaction terms are nested in the list of adjusted main effects, you're kosher. You just need to test all such terms simultaneously.

The benefit of the first approach is that there are fewer degrees of freedom lost by estimating the full model and hence more power for smaller datasets. The benefit of the second approach is that it is more robust and can estimate more sophisticated interactions between parameters. This comes at the cost of a loss of power when, usually, most interactions worth reporting can be estimated with a linear term.

Answer 2

It's not strange at all. If the coefficients on $x^2$ and $z^2$ have the same sign then the regression surface is an elliptical bowl -- open up if the sign is positive, inverted if the sign is negative. If the signs differ then you have a saddle-shaped surface. In either case, you can rotate in the $x,z$ plane to get new predictors $u = x \cos(\theta) + z \sin(\theta)$ and $v = z \cos(\theta) - x \sin(\theta)$ that support a main-effects-only model of the form $y = a + b_u(u - c_u)^2 + b_v(v - c_v)^2$.

With any luck, the new predictors will make some substantive sense.