0
$\begingroup$

I evaluated the effectiveness of an intervention by using lme4 package with the variables

group (intervention=1, control=2=reference group), time (pre-intervention=1=reference group, post-intervention=2)

lmer(Depression ~time *group +(1|id))

resulting in a significant interaction (time * group).

In a second step, I explored if a diagnosis (1=yes/2=no=reference group) has an effect on treatment effectiveness by checking a 3-way interaction with lmer:

lmer(Depression ~diagnosis*group *time +(1|id))

The 3-way interaction (diagnose1:Group1:time2) was not significant. However, the 2-way interaction was no longer significant. In both models, the diagnosis itself is not significant. Regarding interpretation I was advised that the non-significant 2-way interaction corresponds to whether group * time interaction is present in the reference group of the third variable (diagnose =2 -> no diagnose).

Isn't the correct interpretation: it corresponds whether group * time interaction is present in the third variable diagnose (1) compared to the reference category of the third variable diagnose (2)?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I don't have any idea what either what you were advised or what you suggest means. What does "present in the third variable"?

The two way interaction means that the relationship between time and depression is different at the different levels of group, and, likewise, the relationship between group and depression is different at different levels of time. That's good news for your intervention! Assuming the signs of the coefficients are appropriate, it means people got better faster in the group than the control (but do check). The two way interaction doesn't say anything about the third variable.

Also, don't rely too much on significant vs. not. Look at effect sizes and their standard errors. See Andrew Gelman's paper "The Difference Between Significant and Not Significant is not, Itself, Statistically Significant".

Improve this answer
$\endgroup$
5
  • $\begingroup$ Many thanks for your reply. The interpretation of the 2-way interaction is clear to me. I am also aware of the importance of effect sizes. But what about the 3-way interaction? If I add the variable diagnosis (1=yes; 2=no ref group), the 3-way interaction itself is not significant (CI includes 0). This means that diagnosis has no impact on the treatment effect. $\endgroup$
    – Sandi
    Commented Aug 11, 2023 at 8:14
  • $\begingroup$ However, in this model, the 2-way interaction (which represents the treatment effect) is no longer apparent (CI of coefficient includes 0, p-val not significant any longer). How would you interpret this? That there is not enough evidence to claim diagnosis moderates the treatment effect, but there is evidence of a treatment effect (resulting from the 1. model)? Many thanks $\endgroup$
    – Sandi
    Commented Aug 11, 2023 at 8:14
  • $\begingroup$ I would NOT interpret it. That is, I would not pay attention to the fact that it went from significant to not significant. See Gelman's paper. I would look at changes in effect size and in predictions. $\endgroup$
    – Peter Flom
    Commented Aug 11, 2023 at 11:40
  • $\begingroup$ Thanks. Would you evaluate the influence of diagnosis in another way? Would you not add it as a 3-way interaction but only diagnosis as a predictor? Which paper from Gelman exactly do you mean? $\endgroup$
    – Sandi
    Commented Aug 15, 2023 at 13:40
  • $\begingroup$ I would look at effect size, its affect on other parameters, and its substantive meaning. The results of that would determine whether I added it. The Gelman paper is the one I named in my answer "The Difference between Statistically ... "etc. $\endgroup$
    – Peter Flom
    Commented Aug 15, 2023 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.