0
$\begingroup$

I got a research question, where the hypothesis (derived from theory) postulates, that the relationship between predictor X and outcome Y is moderated by W (X+W+X*W -> Y). All variables are sums of likert-scaled items and treated as interval/metric. N=34.

Ignoring stepwise inclusion of variables, I just did a moderation analysis according to the hypothesis and found that everything is hypothesis conform. R-Squared is .20 which is low but expected because other, not included factors are known.

But when I just check the "lower" regressions, i.e. X -> Y, W -> Y and X+W -> Y, then all coefficients are close to zero and none is significant (p-values are incredibly high, p>0.6) and R-Squared is very low also.

What is happening here? Is the expected outcome of the high level regression with interaction term just a statistical artefact? I would like to understand what may be the reasons for this cumbersome result. And also I would like to know if/how I can interpret the results.

EDIT: To avoid misunderstandings: I meant that in the model with interaction term, the coefficients of X, W, and X*W are significant. So there are main effects of X and W as well. But this is cumbersome, because in the model without interaction, neither the coefficient of X nor the coefficent of W are significant and the coefficient values itself are pretty small as you can see in the output:

output

enter image description here

So if there are main effects in the interaction model, these have to appear in the model without interaction as well, isn't it? Maybe in the interaction model, have too much terms regarding the small sample size of N=34. I am still very curious about what's happening here and how I can interpret findings.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

Your results illustrate the importance of interaction terms. This is no "illusion"; this phenomenon might be expected whenever the effect of one predictor on outcome depends on the value of another predictor.

The cookie-yield example in the Wikipedia entry on interactions in statistics provides an easy-to-grasp illustration. The outcome is the yield of "good" cookies (neither overcooked nor undercooked) as a function of oven temperature ("low" versus "high") and time ("short" versus "long") as predictors. In that example, neither temperature nor time--even in an additive model without the interaction--was associated with cookie yield.

Unsurprisingly to anyone who has burned a batch of cookies, however, the interaction term is highly significant. In the no-interaction analysis, the beneficial effect of longer time at the lower temperature was counterbalanced by the deleterious effect of longer time at the higher temperature. Once the possibility of an interaction between temperature and time was considered, however, the implications for cookie yield became clear.

So your hypothesis that moderation was important in your case is supported. If anything, the lack of significance of the predictors in the additive model shows how wise you were to include the interaction term based on your knowledge of the subject matter.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you, but I think you misunderstood me: in your wikipedia example, there's an interaction without main effects. My findings suggest 2 main effects and an interaction effect in the X+W+X*W model, but no effects at all in the simple X+W model. I clarified this in my question and added output values for both models. $\endgroup$
    – Madamadam
    Apr 4, 2020 at 19:13
  • 1
    $\begingroup$ @Madamadam similar situations can expose “significant main effects” that are only seen with an interaction term included. Those really aren’t “main effects“ though; with the interaction those coefficients represent the slope for the indicated predictor when the other predictor has a value of 0. All that “significance” means for those coefficients is that their estimates differ from 0 in that specific case. If there’s an interaction, you need to add the contribution of the interaction term to the contributions from those individual coefficients. More cumbersome, but necessary. $\endgroup$
    – EdM
    Apr 4, 2020 at 20:49
  • $\begingroup$ Thanks again. What you wrote sounds kind of familiar to me, but I'll have to wrap my head around this more profoundly — is there a online source, where I can read more about this? $\endgroup$
    – Madamadam
    Apr 4, 2020 at 22:16
  • 1
    $\begingroup$ @Madamadam see this thread, my answer there, and the links provided in the comments for a start on more details. $\endgroup$
    – EdM
    Apr 4, 2020 at 22:45
1
$\begingroup$

I agree with EdM that this is not an illusion, but rather a demonstration of the importance of considering nonlinear models.

The negative sign on the interaction term slope may be important. The simple slopes implied by your interaction model results are

X: 1.3 - 0.68 W

W: 2.5 - 0.68 X

You don't say whether X and W are centered about their mean. Depending on the ranges of X and W in your data, it is quite possible that the near-zero no-interaction slopes are highly descriptive of the mean values for the simple slopes in the interaction model. If the mean of W is about 2, and the mean of X is about 3.5, you would be right there.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you! X and W weren't centered. X has a mean of 3,09 (min=1, max=4) and W has a mean of 2,06 (min=1,13 max=3,63). In my original question, I added the model with SPSS PROCESS-Macro setting "mean center for construction of products" enabled. Then significance of main effects vanish.Unfortunately, I don't exactly undertstand what this means and what your comment means for my further proceedings … should I give nonlinear models a try? $\endgroup$
    – Madamadam
    Apr 4, 2020 at 22:09
  • $\begingroup$ … or did you mean, my model is quite appropriate, because them mean of W is about 2 and the mean of X is round about 3.6? $\endgroup$
    – Madamadam
    Apr 5, 2020 at 10:57
  • 1
    $\begingroup$ Here is one link to a nice explanation of the concept of simple slopes: quantpsy.org/interact/mlr2_instructions.pdf There is not necessarily a contradiction between the two sets of results. I have no idea whether the interaction model is "correct" or not, but I see no reason to distrust the results, based on what is presented here. $\endgroup$
    – Ed Rigdon
    Apr 5, 2020 at 14:12
  • 1
    $\begingroup$ Thanks. Here's the reason explained why mean centering can change main effects: stats.stackexchange.com/questions/65898/… $\endgroup$
    – Madamadam
    Apr 5, 2020 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.