6
$\begingroup$

Suppose we have a linear regression model with two covariates, $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2$.

There are three possible scenarios:

  1. Both $\beta_1$ and $\beta_2$ are significant.
  2. Either $\beta_1$ is significant and $\beta_2$ is not, or $\beta_2$ is significant and $\beta_1$ is not.
  3. Neither $\beta_1$ nor $\beta_2$ is significant.

My question is, statistically (not theoretically), under which case (scenario) is further investigation of the interaction between $x_1$ and $x_2$ meaningful? For example, investigating a model $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1 x_2$. I have seen many people further investigating interaction effects under scenarios 1 and 2, but I am not sure if we should also try to evaluate the interaction effect for scenario 3.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
12
$\begingroup$

Yes, it makes sense to test for interactions regardless of main effects .

b1 might have a positive slope if b2 is positive, and a negative slope if b2 is negative. That means that the average effect of b1 is 0.

Real life example: When people watch you doing something, it can increase your nervous system arousal. If it's something you're very good at, that might make you better. If it's something you're bad at, it might make you worse. An audience watching people play pool makes the good players better, and the bad players worse. On average, the effect of the audience is zero. That doesn't mean that an audience doesn't have an effect.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Testing the interaction coefficient alone is questionable, though: one ought to test the group of variables with an omnibus F test. See stats.stackexchange.com/questions/65898 for a related discussion. $\endgroup$
    – whuber ♦
    Commented Jan 19 at 19:31
13
$\begingroup$

First, I wouldn't base any sort of decision about model building on the significance of earlier phases of the model building. Statistical significance is tied up with sample size. I would use effect sizes.

But, second, even if both main effect sizes are very close to 0, the interaction can be hugely important. This can happen when the two effects are crossed, as in the example that Jeremy Miles gave in his answer. Or one can dilute the other without the two being crossed.

Third, it can make sense to investigate the interaction because it is part of your hypotheses or research questions. I know you said you wanted something "statistical" not "theoretical" but I don't think that dividing the issues that way makes sense here.

Improve this answer
$\endgroup$
8
  • $\begingroup$ How would you establish a minimal effect size for an interaction? $\endgroup$
    – whuber ♦
    Commented Jan 19 at 19:32
  • 2
    $\begingroup$ Based on substantive knowledge, as with any kind of effect size. Even a small effect could be important in some fields (e.g. airplane manufacturing). $\endgroup$
    – Peter Flom
    Commented Jan 19 at 20:07
  • 1
    $\begingroup$ Substantive knowledge of how big an effect is "big" is common. The statistician can show what different interactions would look like. "Under an interaction like XXX, 100,000 more people are predicted to get pimples." or whatever. $\endgroup$
    – Peter Flom
    Commented Jan 19 at 22:44
  • 1
    $\begingroup$ That sounds like a good approach! But doesn't that imply you're really testing all three parameters (two slopes and the interaction) simultaneously, rather than the interaction alone? Or would you really be asking the expert(s) to compare two sets of predictions: one with and another without the interaction? $\endgroup$
    – whuber ♦
    Commented Jan 21 at 18:48
  • 1
    $\begingroup$ I would present a variety of models. First one with only the main effects. Then ones with different interactions. But always in terms of what the model would predict. The parameters for interactions are, indeed, hard to interpret, even for statisticians! $\endgroup$
    – Peter Flom
    Commented Jan 21 at 21:00
3
$\begingroup$

A large driving force behind modeling interactions is theory, but I realize that your question says you are not looking for that specifically, so I will not dive into that further (though I discuss this in the first part of my answer here for those interested).

To your main question:

My question is, statistically (not theoretically), under which case (scenario) is further investigation of the interaction between $x_1$ and $x_2$ meaningful?

Gelman and his colleagues suggest that “When inputs have large main effects, it is our general practice to include their interactions as well” (p.246-247 in this book). When are the effects large? Plotting the interactions can go a long way to understanding if there is indeed an interaction worth noting. Using the second part of the linked answer above as an example, we can clearly see that the bottom interaction between two continuous variables is minimal and probably not worth investigating further:

enter image description here

Whereas the plot from this answer shows a strong continuous by categorical variable interaction that is probably important to look at further (the left plot doesn't include an interaction in the model, whereas the right plot does). We can see that if we don't include the interaction, we are effectively ignoring the change in the data points and assuming that only the conditional mean varies by group and that the slope is generally negative:

enter image description here

Care must be taken here if you had not considered an interaction model in advance. I would say that this advice can potentially lead to HARKing if its not clearly made explicit a priori that you 1) had a previously conceived main effects-only model first and 2) you tested this only after discovering that the magnitude of main effects is large.

I will add that if you include an interaction, it is my opinion that you should always include both main effects and interactions together. By only entering the interaction, you are not disentangling the independent main effects on the outcome, which may be important to investigate.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.