I am using ANOVA with repeated measures to test significance between males and females results of an experiment during which participants had to evaluate 7 stimuli in 2 conditions (EXP1 and EXP2).
The problem is that even if from results it is clear that there are significant differences between males and females, I don´t get significance in the ANOVA results. Definitively there is an error, because results cannot be non-significant. Indeed looking at the means for each stimulus, it is possible to notice that males gave always higher evaluations than females.
To prove this I discarded for a moment the effect of the repeated measures, and I performed an ANOVA separately on both the two conditions (EXP1 and EXP2) during which the evaluations were given. What I get is significant differences between males and female, in both EXP1 and EXP2.
Now, why when I perform the ANOVA with repeated measures I don´t get the same behavior?
The structure of my table is the following: subject, stimulus, condition, sex, response. The design is the following:
- sex is a between-subjects factor (with two levels)
- stimulus is a within-subjects factor (with 3 assumed levels)
- condition is a within-subjects factor (with 2 levels)
- all factors are fully crossed
Example:
subject stimulus condition sex response
subject1 gravel EXP1 M 59.8060
subject2 gravel EXP1 M 49.9880
subject3 gravel EXP1 M 73.7420
subject4 gravel EXP1 M 45.5190
subject5 gravel EXP1 M 51.6770
subject6 gravel EXP1 M 42.1760
subject7 gravel EXP1 M 56.1110
subject8 gravel EXP1 M 54.9500
subject9 gravel EXP1 M 62.6920
subject10 gravel EXP1 M 50.7270
subject1 gravel EXP2 M 70.9270
subject2 gravel EXP2 M 61.3200
subject3 gravel EXP2 M 70.2930
subject4 gravel EXP2 M 49.9880
subject5 gravel EXP2 M 69.1670
subject6 gravel EXP2 M 62.2700
subject7 gravel EXP2 M 70.9270
subject8 gravel EXP2 M 63.6770
subject9 gravel EXP2 M 72.4400
subject10 gravel EXP2 M 58.8560
subject11 gravel EXP1 F 46.5750
subject12 gravel EXP1 F 58.1520
subject13 gravel EXP1 F 57.4490
subject14 gravel EXP1 F 59.8770
subject15 gravel EXP1 F 55.5480
subject16 gravel EXP1 F 46.2230
subject17 gravel EXP1 F 63.3260
subject18 gravel EXP1 F 60.6860
subject19 gravel EXP1 F 59.4900
subject20 gravel EXP1 F 52.6630
subject11 gravel EXP2 F 55.7240
subject12 gravel EXP2 F 66.4220
subject13 gravel EXP2 F 65.9300
subject14 gravel EXP2 F 61.8120
subject15 gravel EXP2 F 62.5160
subject16 gravel EXP2 F 65.5780
subject17 gravel EXP2 F 59.5600
subject18 gravel EXP2 F 63.8180
subject19 gravel EXP2 F 61.4250
.....
.....
.....
.....
As you can notice each subject repeated the evaluation in 2 conditions (EXP1 and EXP2).
What I am interested in is to know if there are significant differences between the evaluations of the males and the females (both at global level and for each stimulus).
This is the command I used to perform the ANOVA with repeated measures:
aov1 = aov(response ~ sex*stimulus*condition + Error(subject/(stimulus*condition)), data=scrd)
summary(aov1)
Doing so I don´t get significance for the differences between males and females.
Instead if I perform the ANOVA on the two subtables of EXP 1 and 2 I get significant differences.
table_EXP1 <- subset(scrd, condition == "EXP1")
table_EXP2 <- subset(scrd, condition == "EXP2")
fit_table_EXP1 <- lm(response ~ stimulus*sex, data=table_EXP1)
anova(fit_table_EXP1)
fit_table_EXP2 <- lm(response ~ stimulus*sex, data=table_EXP2)
anova(fit_table_EXP2)
How can this be possible? Is it a contradiction?
stimulus
is a within-subjects factor, i.e., each observer gives a response for each stimulus. To get the correct split-plot ANOVA (one between factorsex
, one within factorstimulus
), you have to usesummary(aov(response ~ stimulus*sex + Error(subject/stimulus), data=table_EXP1))
instead of your calls tolm()
. $\endgroup$lm()
formula, you includestimulus
as a factor. If it was within-subjects in your original design, it remains within-subjects in your two subsetted designs. So you still have repeated measures for that factor intable_EXP1
, and intable_EXP2
. $\endgroup$stimulus
factor: hence you have repeated measures with regards to thestimulus
factor, and need to useaov()
with anError()
term. $\endgroup$