Interaction term is significant WITHOUT main affects... AND main effects are significant WITHOUT interaction term?

Question

I am trying to determine the effect of a person's weight and the incline that they are running over on their running speed. I'm just using a simple linear model in R, but I get a weird situation where these two main effects (when viewed without an interaction term) are both significant (and interaction isn't), but when I view the interaction term by itself without main effects, then IT becomes significant! How do I choose between these two conflicting models?

Here's the full model, where neither predictor variable appears significant.

Call:
lm(formula = speed ~ actual.weight * incline, data = wow)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.311468 -0.101650  0.000843  0.092570  0.307654 

Coefficients:
                        Estimate Std. Error t value Pr(>|t|)    
(Intercept)            1.2301738  0.0353404  34.809   <2e-16 ***
actual.weight         -0.0247079  0.0230644  -1.071    0.287    
incline               -0.0004380  0.0005993  -0.731    0.467    
actual.weight:incline -0.0005566  0.0003970  -1.402    0.164    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1288 on 102 degrees of freedom
Multiple R-squared:  0.1859,    Adjusted R-squared:  0.162 
F-statistic: 7.766 on 3 and 102 DF,  p-value: 0.0001011

Since nothing seems to be significant in the full model, I remove the interaction term and see what if things look different:

Call:
lm(formula = speed ~ actual.weight + incline, data = wow)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.31216 -0.10062  0.00313  0.08915  0.31215 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)    1.2618681  0.0272936  46.233  < 2e-16 ***
actual.weight -0.0496668  0.0147356  -3.371  0.00106 ** 
incline       -0.0011274  0.0003442  -3.275  0.00144 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1294 on 103 degrees of freedom
Multiple R-squared:  0.1703,    Adjusted R-squared:  0.1541 
F-statistic: 10.57 on 2 and 103 DF,  p-value: 6.693e-05

However, I have some reason to believe that there might be a lone interaction term without main effects. I tested this, just to be safe, and there was significance!

Call:
lm(formula = speed ~ actual.weight:incline, data = wow)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.30143 -0.09795 -0.00455  0.09431  0.31798 

Coefficients:
                        Estimate Std. Error t value Pr(>|t|)    
(Intercept)            1.1981665  0.0159965  74.902  < 2e-16 ***
actual.weight:incline -0.0008925  0.0001889  -4.726 7.22e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1283 on 104 degrees of freedom
Multiple R-squared:  0.1768,    Adjusted R-squared:  0.1689 
F-statistic: 22.33 on 1 and 104 DF,  p-value: 7.218e-06

These models aren't nested, and I'm really confused how to distinguish between them. How are weight and incline really affecting speed?

Answer 1

To start off, you shouldn't be using backwards selection at all if you believe an interaction effect to be present. If the full model is the model you presumed, then its coefficients are the only interesting ones.

Also note that the results of these models do not conflict with each other: The marginal effects do not have the same interpretation as the main effects. The model without interaction estimates an effect of actual.weight and incline, while the model with interaction estimates an effect of either covariate where the other is equal to zero, and an effect for how a change in one affects the slope of the other.

Lastly, all models explain a little variance in the response variable: Your $\text{R}^2$ ranges from 17% to 19%. That means that even if all presumed effects were significant, they don't have a substantial effect.

With that in mind, there are several things to note about the model coefficients. In the interaction model, the interaction effect and the marginal effects (especially that of incline) are both very small. In the model with only main effects, the effects may be significant, but you should really also consider their effect size, which is again probably less than can be considered relevant, although that depends on the scale at which you measured these variables. Unless you used a very small scale for incline, that means that incline has an almost negligible effect compared to weight.

The last model violates the principle of marginality. You cannot include an interaction effect without the variables it is marginal to. It is therefore of little relevance to the question. But for the sake of completion, note how small the coefficient is. Even if it were a valid model, the effect on speed is very small. This of course depends on the scale at which you measured speed, which you should include in your question. However, since the intercept is rather large compared to the slopes, I don't think knowing the scale will change this answer much.