0
$\begingroup$

I designed an experiment,in which there is one two-level categorical independent variable, and a continuous variable as a covariate. The interaction between the two is not significant, however, when I look at the correlation between covariate and dependent variable at each level of the categorical independent variable. The correlation is significant at one level, but not the other. This pattern shows up repeatedly when I use multiple experimental stimuli, and is consistent with my theory. So can I still interpret the result, even if the interaction effect itself is not significant?

To make it clearer, I know that a significant categorical by continuous interaction means that the slope of the continuous variable is different for one or more levels of the categorical variable.However, I am wondering whether the test for interaction effect takes the significance of the slope into account or not or it just test whether there's a significant difference between the two slope coefficients.

Improve this question
$\endgroup$
6
  • $\begingroup$ When I review papers, and authors try to "let the data speak for itself" in this way, and argue that there are differential effects across groups, but don't actually test for them (i.e. testing for an interaction), I dinged them hard for this. $\endgroup$
    – not_bonferroni
    Commented Jan 9, 2017 at 21:01
  • $\begingroup$ You didn't really answer my question, but thank you. $\endgroup$
    – Yijie Wu
    Commented Jan 9, 2017 at 21:11
  • $\begingroup$ I literally did not answer your question. It was a comment. You're welcome. Regarding "I am wondering whether the test for interaction effect takes the significance of the slope into account or not or it just test whether there's a significant difference between the two slope coefficients." , it just tests whether the two slopes are different. $\endgroup$
    – not_bonferroni
    Commented Jan 9, 2017 at 21:22
  • $\begingroup$ Thanks. When I look at the simple slope, one is significant, and another is not. The non-significant slope would mean that this slope coefficient is no different from zero. Then doesn't that mean despite the significant interaction, the continuous independent variable has no effect on the dependent variable on one level of the categorical independent variable? $\endgroup$
    – Yijie Wu
    Commented Jan 9, 2017 at 21:49
  • 1
    $\begingroup$ The non-significant slope means that you can't say conclusively the slope is not 0. It does not mean it is no different from 0, $\endgroup$
    – David Lane
    Commented Jan 9, 2017 at 22:18

1 Answer 1

Reset to default
1
$\begingroup$

A correlation being significant at one level but not the other does not justify the conclusion that the slopes (or correlations) are different. What if one were significant at 0.049 and the other were not significant with p = 0.051. You wouldn't be in a position to say they were different.

A broader point is that your results can still support your theory even if the effect is not significant although the support will not be conclusive. You could write something like "As predicted, the slope was larger for condition A (b= ) than for condition B (b= ). However, this difference did not reach conventional levels of significance (p=.068) so the evidence for a difference is suggestive but not conclusive. Depending on your results you could say "strongly suggestive," "weakly suggestive," etc.

Sorry if this approach offends those in the Newman-Pearson school. Please read the first few pages of this if you believe the null hypothesis should either be rejected or not rejected.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.