7
$\begingroup$

Lets say I have a linear regression with two numeric explanatory variables: A and B.

Consider the following scenarios:

  1. A and B are both insignificant
  2. A is significant, B is insignificant; or the other way around
  3. A and B are both significant

Now, my question:

In which scenarios is it possible (or should we differentiate between "theoretically possible" and "likely" here?) that the interaction term A * B is going to be significant?

$\endgroup$
15
$\begingroup$

$A*B$ can be significant in all of these scenarios. Consider $A \in \{-1, 0, 1\}$ and $B \in \{-1, 1\}$ where the underlying model is $E[Y|A,B] = A*B$. In a roughly balanced situation, with (roughly) equal sample sizes for each combination of $A \times B$, neither $A$ nor $B$ will be significant (except for the $\alpha$ fraction of the time when a true null hypothesis is rejected), but the interaction term certainly will be! Here's a numeric example:

A <- rep(c(-1,0,1), 100)
B <- rep(c(-1,1), 150)
X <- A*B
Y <- X + rnorm(300)

> summary(lm(Y~A+B+A*B))

Call:
lm(formula = Y ~ A + B + A * B)

Residuals:
     Min       1Q   Median       3Q      Max 
-3.03520 -0.59349 -0.03184  0.62857  2.49359 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.02083    0.05668  -0.367    0.714    
A           -0.03797    0.06942  -0.547    0.585    
B            0.05867    0.05668   1.035    0.301    
A:B          0.90789    0.06942  13.078   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.9818 on 296 degrees of freedom
Multiple R-squared: 0.3681, Adjusted R-squared: 0.3617 
F-statistic: 57.47 on 3 and 296 DF,  p-value: < 2.2e-16 

Or, more simply:

> cor(A,Y)
[1] -0.02527534
> cor(B,Y)
[1] 0.04782935
> cor(A*B,Y)
[1] 0.6042723

It should be intuitively clear that if we can construct an example where $A$ and $B$ are both insignificant, yet the interaction is significant, we can do so for either of your other two cases.

As for likely... One could argue that in real life, apart from physics and a few other disciplines, pretty much all interaction terms are very likely to be nonzero (albeit perhaps very small), and "significance" in its statistical sense is merely a function of sample size.

$\endgroup$
  • $\begingroup$ Thanks, this is an excellent answer! If I would have rephrased "significant" to "has explanatory power", would the answer be different? My intuition tells me that if A cannot explain Y, and B cannot explain Y, then A * B cannot explain Y either (or this is extremely unlikely). Basically I have a whole set of variables, of which many are insignificant. I would like to generate a set of interaction variables that are likely to have an impact, and was hoping that I could leave out all insignificant explanatory variable sets. $\endgroup$ – Tom Feb 12 '12 at 14:46
  • 5
    $\begingroup$ I'm not so sure about that; in many cases, once you include an interaction term, A and B become unimportant. A simple example is estimating the weight of a tree based on its height and diameter at its base; once you multiply the two terms, the individual terms will drop out (although w/o the interaction, they would both be significant.) Often you need more than one gene for a trait to express itself (e.g., two blue-eyed parents for a blue-eyed child), or you need a gene and an environmental factor together for an effect to occur. Not so sure about the social sciences, though... $\endgroup$ – jbowman Feb 12 '12 at 15:05
  • $\begingroup$ @jbowman Hi, I also got such problem in a project last year, but I thought in such case Y=A*B, it seems that using log transformation for both sides lm(log(Y)~log(A)+log(B)) is more suitable, with less terms, variance. And the diagnostic plots are more "nice" (valid), while the residual plots of Y~A+B+A:B is still concave like. But in my project, some maths logic can be shown to use log for both sides. What if other project without too much prior knowledge? Forward selection seems may not be able to include A:B sometimes in such case, while backward must be able. $\endgroup$ – Vincent Nov 1 '13 at 17:01
9
$\begingroup$

Jbowman's answer is correct but to add to the "real life" dimension he or she adverts to: You really should think about "real life" here because the basic answer to your question is: "Impossible to say; it depends on what you are modeling."

The answer to the main question -- can there be a "significant" interaction between two "nonsignificant" predictors -- is "of course."

Imagine, e.g., a disease that is equally likely to be terminal for members of two subpopulations & that can be effectively treated with an intervention in only 1. Membership in the groups will not predict death from the disease; and the main effect of the treatment -- which will be a (sample-size weighted) average of the effect on the two groups might well be nonsignificant too if the sample size of the treatment-responsive population or the effect size of the intervention is small. But add a cross-product interaction term -- & voila, you see that the effect of treatment is "significant" for the treatment-responsive group.

Maybe you can see from this example that your questions about the relative "likelihood" & "theoretical possibility" etc. of signficant interactions conditional on the predictor & moderator being significant can't be answered in a meaningful way. Everything depends on how the predictor & moderator are related to the outcome being modeled.

For a phenomenon in which it is not meaningful or plausible to envision the two variables interacting, there's no point asking about how "likely" or "theoretically possible," whether or not the predictor and moderator are significant or nonsignificant (likely the interaction will be nonsignificant in that case, but if it turns out otherwise, it's likely a coincidence or a reflection of "significant" but meaningless relations between variables when you have large sample, etc.)

If such a relationship is plausible, then by definition a "significant" interaction is "theoretically possible" & whether one would expect the predictor and moderator to be significant or nonsignificant on their own in that situation necessarily depends on what you are modeling. (Because the universe of things you might investigate is infinite, there's no way to say what's more likely -- both variables, one, or neither "significant" )

Statistics won't help anyone who doesn't known what & why he or she is using them to understand a particular phenomenon.

$\endgroup$
3
$\begingroup$

Alternatively, you can test if the interaction is spurious or not by FWL orthogonalization, and when the interaction stands, then you can remove those independent variables that now are not significant anymore.

The objective is to remove as much interaction as possible because it confuses the analysis of the parameters.

See: Empirical Economics 2012, Hatice Ozer Balli and Bent E. Sørensen, Interaction effects in econometrics. [DOI]

$\endgroup$
0
$\begingroup$

Of course. Let me try and explain in a theoretical way instead of difficult numerical ways.

Let's imagine Psychology research in which you investigate the effect of identification with your ethnic group and the group's attitudinal norm towards an outgroup on the identification with that specific outgroup: $\text{Ingroup ID} + \text{norm} = \text{Outgroup ID}$

Now imagine that only those who highly identify with their ingroup identify strongly with the outgroup IF the attitudinal norm towards that outgroup is high (positive) and very little if the norm towards that outgroup is low (negative).

This is a part of your sample that can result in a significant effect but might be too small if you only check both main effects in which all participants (regardless of specific score on both variables) are put into the equation to see for any effect on the dependent variable.

In other words, all the scores taken together without any specific interacting combination nullifies a significant effect.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.