3
$\begingroup$

I am conducting a psycholinguistic experiment. Each trial consists of the subject responding to a word by pressing a button.The design of my experiment is as follows:

5 blocks of a 100 trials each (each trial is a response). In each block, 50 trials are Regular and 50 are Random. Response time (RT) is the dependent variable. This experiment is partly a learning experiment. I expect RTs to be faster in the Regular condition because there is learning which allows for faster responses.

I therefore want to test the difference in mean response time (RT) between the two conditions of my experiment.I know that the model for the ANOVA would be something like 

RTs ~ Type of Trial (`Regular` and `Random`) + ...

I have two related questions

  1. Should I use Trial or Block as the other factor? (or both?) Normally I would compare mean RT for each condition across blocks, but it is possible that within blocks, there is already changes in RT between trials. So, is Block, Trial or both my factor? (besides the condition factor).

  2. Since I expect variation across participants, should I include Subject as an Error variable in the model?

I am doing my analysis in R, in case anyone wants/can provide some advice in that format, but also general statistical advise is much appreciated.

Improve this question
$\endgroup$
2
  • $\begingroup$ Do you expect the difference between “regular” and “random” to change over the time of the experiment? What do you want to test? If there is indeed a difference between types of trials? $\endgroup$
    – Gala
    May 6, 2013 at 6:28
  • $\begingroup$ @GaëlLaurans Yes I expect the difference in RT between Regular and Random to change over time. Specifically, both trials will become faster as time progresses, but the decrease in RT for Random trials should be a linear decrease based on practice effect, whereas the decrease in RT for Regular trials should be steeper because of the learning effect. Answering your second question, I want to test whether the differences in RT are different. So to answer your third question, there is indeed a difference between types of trials. The difference is in fact the experimental manipulation. $\endgroup$
    – HernanLG
    May 6, 2013 at 7:50

1 Answer 1

Reset to default
2
$\begingroup$

A simple way to analyze this data set would be to average response times in each condition across all blocks and trials, as you usually do. It is less than ideal and you will loose power but it would still provide a test of your hypothesis. Alternatively, you could focus only on the last block. You would be throwing away a lot of data but 50 trials should be enough to get a good measure of response time and any evidence of a difference between conditions would support your hypothesis. Both of these analyses could be carried out with a simple paired sample t-test.

Another similar trick would be to treat this as a pre/post design, using only the first and the last block. You can look at Best practice when analysing pre-post treatment-control designs for some advice on the best way to analyze those.

Including Block as a factor in a repeated measures ANOVA is problematic. The main issue is that, if I understood you correctly, you expect performance to change relatively smoothly over time (perhaps linearly, possibly leveling off after some time or following a logistic curve) so that subsequent blocks would be more similar to each other than to other blocks. This is not captured by a repeated measures ANOVA and would lead to a violation of the sphericity assumption. All this would be even worse for Trial, which suffer from an additional problem: Trials are nested within conditions and blocks so that you cannot include all three in a regular ANOVA model.

If you want to go beyond that and actually model the change across individual trials, I think your best bet would be to look at multilevel models. In fact, if you use a different set of words in each condition, all the models discussed until now lead to questionable inferences because of what is known in psycholinguistics as “the language-as-fixed-effect fallacy”. The following reference describes the problem and some ways to deal with it. Even if it turns out that it is not an issue in your case, you can use it as an entry point into the literature on multilevel models.

Baayen, R.H., Davidson, D.J., & Bates, D.M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59 (4), 390-412.

Considering all this, I think your second question is not relevant anymore but if you want to fit repeated measures ANOVA models, you would use things like:

RT ~ Type + Error(Subject/Type)
RT ~ Type*Block + Error(Subject/(Type*Block))
Improve this answer
$\endgroup$
1
  • $\begingroup$ This is very helpful, many thanks for the clarifications and suggestions. $\endgroup$
    – HernanLG
    May 6, 2013 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.