# Testing a 2 by 2 mixed ANOVA in R

I have data from an experiment that is testing how the order of two studying methods (visual or auditory) affects word recall. For analysis a multi-factor anova with a repeated measure is appropriate, but I am not sure if I am structuring my data correctly.

This is the command I'm using: aov(recalled_items~task*order)+Error(subject/task)+(order))

Here is an example of the data structure:

Subject    Task    Order    Recalled Items
A        Visual    First       13
A       Auditory   Second      22
B        Visual    First       14
B       Auditory   Second      28
C        Visual    Second      10
C       Auditory   First       15
D        Visual    Second      14
D       Auditory   First       29

• Does R know to compare Visual 1 and Visual 2 recall values and Auditory 1 and Auditory 2 recall values?

I am worried that because of the way I structured my data R is just comparing Visual 1 and Auditory 1 and as a result I am getting no effect.

• I tweaked the question title to make it a little more specific on the assumption that "order" is a between subjects factor. Feel free to change back if I've misconstrued? May 30, 2011 at 15:08
• If order is a between-subjects factor, it should be constant for each subject - it could have levels VisualFirst and AuditoryFirst. Your call to aov() looks odd, did you mean aov(recalled_items ~ task*order + Error(subject/task))? May 30, 2011 at 15:18
• If your data represent number of items recalled of some total number of items presented, it would be better to represent your data in its raw binary format, labelling each item as remembered or not remembered and conducting a mixed effects model while specifying binomially distributed error. May 30, 2011 at 15:44
• @caracal Do you need to calculate an error term for a between subject factor or not? Jun 9, 2011 at 17:39
• No, just the within-factor. The parentheses in your call to aov() don't match, and I don't understand the + (order) at the end. Jun 10, 2011 at 13:27

Your method won't work because it's going to treat order within. Try this...

Subject    Task    Order    Recalled Items
A        Visual    vFirst      13
A       Auditory   vFirst      22
D        Visual    aFirst      14
D       Auditory   aFirst      28


Or something like that.

• Thanks. I realized after I posted that this would be the correct way to enter the data. So I agree with this change. Jun 9, 2011 at 16:29

Assuming that John's correction to the data is correct, you can use the ezANOVA function from the ez package:

my_anova = ezANOVA(
data = my_data
, dv = .(recalled_items)
, wid = .(Subject)
, between = .(Order)
)


print(my_anova)

However, as I note in a comment below your question, it looks like your data represents the number of items recalled from a list of study items, in which case it would be more appropriate to analyze the raw item-by-item recall data. See my answer to this question for a description of how to achieve this. That answer explicitly speaks to recognition memory type scenarios where one typically wants to dissociate response bias from discriminability. If your context if free recall, then you would simply have data labelling each studied word as recalled or not-recalled and this would be your response variable. You would then predict this response as a function of the explanatory variables in your study and their effect would represent the change in likelihood (on the log-odds scale) of recall.

• Hi Mike, Thanks for your help. The actual independent variable that I am measuring is the magnitude of an evoked response measured with EEG. I was trying to make it simple to avoid extra explanations about the data. I am also using a specific set of stimuli that I am including in the analysis (a unique value for each stimulus) as a within subject factor. I think this should be OK. Jun 9, 2011 at 16:51