0
$\begingroup$

In my study there were 63 types of stimuli (63 levels of the IV) and all types had 8 different examples (8 trials per level of IV) which were all categorized by each participant. Accuracy of categorization was the dependent variable.

I originally arranged my data for R such that all 8 responses for a stimulus type were averaged into a single accuracy score by participant. Simplified example follows:

Table Example 1

I assumed, incorrectly, that I would get the same results in a repeated measures ANOVA by arranging my data such that each stimulus example of a stimulus type constituted its own row (I am now analyzing my results with this arrangement instead of the old one). Simplified example follows:

Table Example 2

When I ran my repeated measures ANOVA in R I got different F scores as well as different results from paired comparisons for each way I organized the data. I know that there must be a difference in how variability is calculated between the two data arrangements, but I don't understand why. Can someone give me some more insight in to why the results differ?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ What is your factor ? And what model do you apply to the averaged data ? $\endgroup$
    – Stéphane Laurent
    Commented Oct 3, 2014 at 18:50
  • $\begingroup$ It would be helpful to see the code that you used to fit the model in R. You should be using the latter setup. The model you should be using is a GLMM or a GEE logistic regression. But I wonder if you were using OLS (ie lm()). $\endgroup$
    – gung - Reinstate Monica
    Commented Oct 3, 2014 at 19:00
  • $\begingroup$ My factor in the above examples is stimulus type (emotion). I did a repeated measures ANOVA of the data (not sure if that answers your question about what model I used) Here is the code I used in R: aov.ex <- aov(data$Accuracy ~ data$Stimulus_Type + Error(factor(data$Participant))) $\endgroup$
    – phisher
    Commented Oct 5, 2014 at 20:14

1 Answer 1

Reset to default
0
$\begingroup$

To answer your specific question, the reason the two approaches are giving different results is that you are parsing the variance differently in the two approaches. (To see this, look at the numerator and denominator of the 2 F ratios, and the associated mean squares and df.) In general, there can be more than one logical way to parse the variance in a research study, such as with or without blocks. Beyond this specific answer, under some circumstances you might expect more (or fewer) differences between the two approaches. For example, it may be that your 8 examples are not fungible. It may be that one approach has more variance and is driving the single accuracy score. Have you looked at the average difficulty of each of the 8 examples? More importantly, if the 8 examples are unique to each of the 63 levels of the IV, then there is no basis for parsing out the variance due to examples. If you have 8*63 examples, the overall accuracy approach seems most appropriate. Parenthetically, did you set up your original analysis as a repeated measures approach (because your are collecting data on 63 types of stimuli from each participant)?

Improve this answer
$\endgroup$
1
  • $\begingroup$ The 8 examples (models) are quite different from one another. Some models were, across levels of the IV, more accurately recognized than other models, so they are generally not good substitutes for one another (I think that is what you meant by fungible). Yes the original analysis was set up as a repeated measures. Each participant saw all stimuli. $\endgroup$
    – phisher
    Commented Oct 5, 2014 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.