We have 2 continuous predictors in a regression model, and both have significant quadratic terms. Our main hypothesis we'd like to test is (generally) whether there is an interaction between these variables. When modelling, we see a significant interaction between the linear terms. But interactions terms involving the quadratic terms are not signficant.

So are there any problems with fitting the model with the quadratic main effects and only first order interaction terms (e.g. $y = intercept + x^2 + x + xz + z + z^2$)? I don't know of any theoretical problems with this, but looking at the predicted outcome, this seems to force a strange relationship -- specifically, looking at curves of $y$ versus $x$, for all values of $z$ the curves intersect at a specific point.

In short, the question is, if you are set on including a quadratic covariate, are you then only interested in testing interactions of the highest ordered terms (and if only linear interactions are significant then this means there is no interaction.

  • 2
    $\begingroup$ Does the model have a constant term? $\endgroup$ – Ray Koopman Nov 13 '13 at 0:03
  • $\begingroup$ Yes! sorry, I just edited the original text. Essentially, any interaction terms with quadratic terms are missing, but everything lower order is there. $\endgroup$ – rjweyant Nov 13 '13 at 3:15

Firstly, it's perfectly sane to fit the model given by the following parameterization (I'll call this the "first approach"):

$$ E[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 + \gamma X W $$

and you can use the 1 dimensional Wald test for $\gamma = 0$.

However, the "full model" you described is peculiar $$ E[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 + \gamma_1 X W + \gamma_2 X^2W^2 $$

when, as I read it, you should have instead fit the following model (we'll call this the "second approach"):

$$ E_{full}[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 + \gamma_1 X W + \gamma_2 XW^2 + \gamma_3 X^2W + \gamma_4 X^2 W^2 $$

tested against the non-interaction effect models:

$$ E_{null}[Y|X,W] = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_3 W + \beta_4 W^2 $$

your nested model test is the joint test of $\gamma_1, \gamma_2, \gamma_3, \gamma_4 =0 $. If you reject this null hypothesis, you conclude there is an interaction. But 4 terms is complex.

As long as the interaction terms are nested in the list of adjusted main effects, you're kosher. You just need to test all such terms simultaneously.

The benefit of the first approach is that there are fewer degrees of freedom lost by estimating the full model and hence more power for smaller datasets. The benefit of the second approach is that it is more robust and can estimate more sophisticated interactions between parameters. This comes at the cost of a loss of power when, usually, most interactions worth reporting can be estimated with a linear term.

| cite | improve this answer | |
  • $\begingroup$ I did an LRT on the 4 terms and this was significant, so my concern is resolved, but I'm still curious about the general problem. It seems that we include all lower order terms/main effects when fitting interactions and polynomials to avoid forcing the function through the origin, or another unnatural 0. When we fit a quadratic function and only a linear interaction, it forces an unnatural point where all the curves will cross. So while there might not be theoretical problems with this interaction, is it reasonable to discount it due to what it means within the problem? $\endgroup$ – rjweyant Nov 14 '13 at 14:53
  • $\begingroup$ Well, if statistical significance alone resolves your concerns, I am now concerned for other reasons! As long as the interaction term(s) is/are nested in the product of main effects, then the interpretation of the test is conserved, albeit more difficult. With quadratic effects, the interpretation of the 1st order effect is the instantaneous rate of change at the apex of the parabola. I'd have to sit down with a 3d plotter to really see what the differences are btn these various modeling approaches. $\endgroup$ – AdamO Nov 14 '13 at 19:06

It's not strange at all. If the coefficients on $x^2$ and $z^2$ have the same sign then the regression surface is an elliptical bowl -- open up if the sign is positive, inverted if the sign is negative. If the signs differ then you have a saddle-shaped surface. In either case, you can rotate in the $x,z$ plane to get new predictors $u = x \cos(\theta) + z \sin(\theta)$ and $v = z \cos(\theta) - x \sin(\theta)$ that support a main-effects-only model of the form $y = a + b_u(u - c_u)^2 + b_v(v - c_v)^2$.

With any luck, the new predictors will make some substantive sense.

| cite | improve this answer | |
  • $\begingroup$ OK, so if I understand correctly, $\theta$ is the angle of the point (x,z) ? And $c_u$ and $c_v$ are the means of the new predictors? $\endgroup$ – rjweyant Nov 13 '13 at 20:56
  • $\begingroup$ And the "strangeness" was not that the intersection point of the curves was unexpected based on the function, but is this an unnatural way to model the problem by forcing the intersection. $\endgroup$ – rjweyant Nov 13 '13 at 21:04
  • $\begingroup$ You're rotating the reference frame in the predictor plane, from $x$ and $z$ to $u$ and $v$. $\theta$ is the angle that the $u$-axis makes with the $x$-axis and the $v$-axis makes with the $z$-axis. Such rotations do not change the general form of the model. The trick is to choose $\theta$ so that the coefficient on $uv$ is zero. This is equivalent to getting the eigenvectors of the 2 $\times$ 2 matrix $((b_{xx}, b_{xz}/2),(b_{xz}/2, b_{zz}))$ whose elements are the coefficients in the original model. $c_u$ and $c_v$ are the locations of the extrema of $y$ on the $u$ and $v$ axes. $\endgroup$ – Ray Koopman Nov 14 '13 at 1:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.