controlled experiment we have 5 groups:
- 2 Control Groups (CG, passive and active)
- 3 Experimental Groups (EG, Doseage of the Intervention).
We also have two measurement time points T1 and T2 and found a significant interaction with a mixed ANOVA between Condition (Group) and time point, with the two control groups basically running in a horizontal line and being n.s. in the post-hoc comparisons.
We want to analyze if the moderator self-efficacy (which we assessed with a questionnaire at T1) moderates the effect of the 3 interventions (experimental groups, EGs). We used the Process Macros SPSS (Hayes) for this.
- First we built a difference score (DS = Dependent Var at T2 - Dependent Var at T1) and calculated model 1 with group membership (5 levels) as predictor (x) and self-efficacy as moderator (w).
- The result for the interaction term (moderation) came back non-significant, which is absolutely fine.
However, one of my fellow m.sc. candidates suggested if it even makes sense to include the two CG in this analysis. Because if there is no effect in those two groups (as indicated by the post-hoc pairwise comparisons) it does not seem logical if the non present effect is moderated by another variable.
- Now indeed if we exclude participants from the two CG and run the analysis just with the 3 EG we get a p of < .001.
Question: is it acceptable or justifiable in an RCT Design, to only include EG (or groups with significant effects in the pairwise comparisons after mixed ANOVA) in the moderation analysis?