# What does it mean when in a mixed design the "within" subjects is significant only when the "between*within" interaction is excluded from the model?

I have a mixed-design, 1 between (group A and group B) and one within (pre and post). Anova, as well as mixed-effects model, showed non-significant results in the between, in the within, nor in their interaction. Descriptives and plots confirmed no differences between groups, but it seems that there are differences between pre and post (i.e., within level).

Edit 1: I've just realised that if I only test the main effects (between + within), and not their interaction (between*within) the within is significant. So the within factor is not significant because of the interaction tested (but the interaction is not significant).

Why is this happening? There are no differences at all between groups, and the starting point for the within variable (Time 1) is similar between groups. Still, "time" is not significant when including the interaction.

Edit 2: I am now wondering if this is a problem related to the small sample size, where the interaction did occur but there is not enough power to detect this effect (while in the boxplots the interaction is clear; in fact in one group 1 there are differences between times, while there are no differences in the other group). Are there any reference I could used to support my reasoning related to small sample and lack of power? Also, when I filtered the data, t-test and Wilcox confirmed that the the effect of time occurred only in one group.

Edit 3: I've got a bayes factor of 2.97999 for the effect of time + id, using anovaBF  function, setting the model as follows:

anovaBF(dependent ~ time*exp_group + id, data = df, whichRandom = "id", progress=FALSE)


These are descriptive:

Group 1, Time 1 = mean (4.48), SD (1.26)
Group 2, Time 1 = mean (4.14), SD (1.42)

Group 1, Time 2 = mean (3.54), SD (1.78)
Group 2, Time 2 = mean (3.61), SD (2.11)


Here the boxplot of the interaction, the dots in the boxex are the means.

• It sounds like you are correct that what you are seeing is related to sample size. Specifically, that when you include the interaction term you are decreasing the residual degrees of freedom and losing power for the testing of other terms. It's difficult to know without seeing the data. Jul 30 at 17:23
• Thank you @SalMangiafico. I've just included the boxplot. I did run a power analysis and do not have the power - but this is only a pilot study. Still, I don't want to present biased results. What references I could use to sustain my argument? Thank you again. Jul 30 at 17:51

Your pilot data have identified a potential difference between the 2 Groups in post-pre differences. You can show your data analyzed in several different ways (e.g., the full interaction model, the model without the interaction, the separate models for the 2 Groups), as you are still in an exploratory phase of the project. You also can show what you found about the clock-time differences in post-pre values as something that needs (ideally) to be controlled directly or at least accounted for statistically.