4
$\begingroup$

I am using ANOVA with repeated measures to test significance between males and females results of an experiment during which participants had to evaluate 7 stimuli in 2 conditions (EXP1 and EXP2).

The problem is that even if from results it is clear that there are significant differences between males and females, I don´t get significance in the ANOVA results. Definitively there is an error, because results cannot be non-significant. Indeed looking at the means for each stimulus, it is possible to notice that males gave always higher evaluations than females.

To prove this I discarded for a moment the effect of the repeated measures, and I performed an ANOVA separately on both the two conditions (EXP1 and EXP2) during which the evaluations were given. What I get is significant differences between males and female, in both EXP1 and EXP2.

Now, why when I perform the ANOVA with repeated measures I don´t get the same behavior?

The structure of my table is the following: subject, stimulus, condition, sex, response. The design is the following:

  1. sex is a between-subjects factor (with two levels)
  2. stimulus is a within-subjects factor (with 3 assumed levels)
  3. condition is a within-subjects factor (with 2 levels)
  4. all factors are fully crossed

Example:

subject  stimulus condition sex        response
subject1    gravel  EXP1    M      59.8060
subject2    gravel  EXP1    M      49.9880
subject3    gravel  EXP1    M      73.7420
subject4    gravel  EXP1    M      45.5190
subject5    gravel  EXP1    M      51.6770
subject6    gravel  EXP1    M      42.1760
subject7    gravel  EXP1    M      56.1110
subject8    gravel  EXP1    M      54.9500
subject9    gravel  EXP1    M      62.6920
subject10   gravel  EXP1    M      50.7270
subject1    gravel  EXP2    M      70.9270
subject2    gravel  EXP2    M      61.3200
subject3    gravel  EXP2    M      70.2930
subject4    gravel  EXP2    M      49.9880
subject5    gravel  EXP2    M      69.1670
subject6    gravel  EXP2    M      62.2700
subject7    gravel  EXP2    M      70.9270
subject8    gravel  EXP2    M      63.6770
subject9    gravel  EXP2    M      72.4400
subject10   gravel  EXP2    M      58.8560
subject11   gravel  EXP1    F      46.5750
subject12   gravel  EXP1    F      58.1520
subject13   gravel  EXP1    F      57.4490
subject14   gravel  EXP1    F      59.8770
subject15   gravel  EXP1    F      55.5480
subject16   gravel  EXP1    F      46.2230
subject17   gravel  EXP1    F      63.3260
subject18   gravel  EXP1    F      60.6860
subject19   gravel  EXP1    F      59.4900
subject20   gravel  EXP1    F      52.6630
subject11   gravel  EXP2    F      55.7240
subject12   gravel  EXP2    F      66.4220
subject13   gravel  EXP2    F      65.9300
subject14   gravel  EXP2    F      61.8120
subject15   gravel  EXP2    F      62.5160
subject16   gravel  EXP2    F      65.5780
subject17   gravel  EXP2    F      59.5600
subject18   gravel  EXP2    F      63.8180
subject19   gravel  EXP2    F      61.4250
.....
.....
.....
.....

As you can notice each subject repeated the evaluation in 2 conditions (EXP1 and EXP2).

What I am interested in is to know if there are significant differences between the evaluations of the males and the females (both at global level and for each stimulus).

This is the command I used to perform the ANOVA with repeated measures:

aov1 = aov(response ~ sex*stimulus*condition + Error(subject/(stimulus*condition)), data=scrd)
summary(aov1)

Doing so I don´t get significance for the differences between males and females.

Instead if I perform the ANOVA on the two subtables of EXP 1 and 2 I get significant differences. 

table_EXP1 <- subset(scrd, condition == "EXP1")
table_EXP2 <- subset(scrd, condition == "EXP2")


fit_table_EXP1 <- lm(response ~ stimulus*sex, data=table_EXP1) 
anova(fit_table_EXP1)


fit_table_EXP2 <- lm(response ~ stimulus*sex, data=table_EXP2) 
anova(fit_table_EXP2)

How can this be possible? Is it a contradiction?

Improve this question
$\endgroup$
13
  • $\begingroup$ You stated that stimulus is a within-subjects factor, i.e., each observer gives a response for each stimulus. To get the correct split-plot ANOVA (one between factor sex, one within factor stimulus), you have to use summary(aov(response ~ stimulus*sex + Error(subject/stimulus), data=table_EXP1)) instead of your calls to lm(). $\endgroup$
    – caracal
    May 22, 2011 at 14:35
  • $\begingroup$ Hi, thanks for your answer. I don´t understan why you keep considering the repeated measures, indeed in table_EXP1 there are not repeated measures. Could you please explain me why you chose this approach? Anyways using your method my problem is still unresolved: the ANOVA result say that the differences are not significant. This is not possible. Indeed as I told, on 7 stimuli, in both the 2 conditions I always get values higher for the males...and are not values very near to those of the females. Any suggestion? Thanks! $\endgroup$
    – L_T
    May 22, 2011 at 15:14
  • $\begingroup$ @user4701 I don't quite follow: In your question, you state "stimulus is a within-subjects factor". In your lm() formula, you include stimulus as a factor. If it was within-subjects in your original design, it remains within-subjects in your two subsetted designs. So you still have repeated measures for that factor in table_EXP1, and in table_EXP2. $\endgroup$
    – caracal
    May 22, 2011 at 18:05
  • $\begingroup$ Hi caracal, thanks a lot again for you answer. The thing is that, according to my (poor) knowledge, the table_EXP1 does not contain repeated measures. Indeed the each subject gave for each stimulus only one answer. So, having precised this, am I still wrong if I use this command? fit_table_EXP1 <- lm(response ~ stimulus*sex, data=table_EXP1) anova(fit_table_EXP1) $\endgroup$
    – L_T
    May 22, 2011 at 22:10
  • 1
    $\begingroup$ @user4701 Yes, still wrong: the point is that one subject not only sees one single stimulus, but EACH one of the stimuli. A single subject is observed in all levels of the stimulus factor: hence you have repeated measures with regards to the stimulus factor, and need to use aov() with an Error() term. $\endgroup$
    – caracal
    May 22, 2011 at 22:19

1 Answer 1

Reset to default
1
$\begingroup$

Apparently the answer is "check your data structure". To that I would also add check the type of the Sums of Squares you are using. Further, I would add, ezANOVA is a godsend in that it handles the model formulation for you and gives more traditional (Type-III Ss) results.

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ I think answers like these belong in the comment section. thanks for being nice @caracal. $\endgroup$
    – suncoolsu
    Aug 17, 2011 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.