What kind of statistical test would I run to find the linear effect of time within sessions for the experimental group (condition=train) in a study that has ten time points (run1-run10) per subject and a DV of interest (reaction time) collected at each time point?

To make the question more concrete, here is an example of the data in R:

       ID condition  time   reaction_time 
1      1    train    run1   200.94         
2      2    train    run1   543.94         
3      3    train    run1     NA         
4      4    train    run1   443.08         
5      5    train    run1   539.38         
6      1    train    run2   433.00         
7      2    train    run2   392.63         
8      3    train    run2   427.83  
  • 3
    $\begingroup$ I am no expert, but for terminology: You would first decide on a statistical model for the data, and then estimate the model parameters, and that is the point where you would use a statistical test to evaluate the size/significance of any effect. You want to treat the run number as an (integer-)time? Does "condition" play a role? (e.g. is there a control group you are comparing the experimental group to?) Do you want to treat the time-effect as per-ID? Or across all ID's together? $\endgroup$ – GeoMatt22 May 13 '17 at 0:32
  • 1
    $\begingroup$ Where you say "find the effect", that's an estimation problem (you're estimating an unknown population effect from data), not a testing problem. $\endgroup$ – Glen_b -Reinstate Monica May 13 '17 at 1:07
  • $\begingroup$ Thank you for the correction of terminology, @GeoMatt22. There is a control group, but for now, I want to look at the learning effect in the "train" group only on a task that is repeated 10 times. I'm trying to look at the time effect across all ID's together at each time point. $\endgroup$ – kdestasio May 13 '17 at 1:12
  • $\begingroup$ This isn't really a question just about how R works. There is a clear statistical question here. I see no reason to think this is off topic. I'm voting to leave open. $\endgroup$ – gung - Reinstate Monica May 13 '17 at 12:06

This could be fit with a multilevel model: Observations are nested within people. Reaction time (the DV) and time (1-10) are both measured at level 1 (i.e., observation).

First, I would format your data such that ID is a factor() and time is numeric() (i.e., 1, 2, 3, etc., instead of run1, run2, run3, etc.)

Second, the packages you want to use are the lmerTest and lme4 packages. The lme4 package authors want you to do nested model comparisons and likelihood ratio tests to see if effects are significant, but an easier way to do this is using Satterthwaite approximation for the degrees of freedom, which is what lmerTest does.

Just like the lm() function in R, you can use the lmer() function from lme4. You specify an equation and the dataset. The only difference is that you will now specify the random effects structure in the formula. If you are only interested in time (i.e., you are ignoring condition), your formula would look like:

lmer(reaction_time ~ time + (1+time|ID), yourdata)

The first part tells us that reaction_time is going to be predicted by time. The part in parentheses tells us that we are allowing each participant (ID) to have their own intercept (1) and slope (time). You can call summary() on that lmer() object to look at the significance of time as well as how much variance there is in both slopes and intercepts.

I would suggest checking out this book or this book as well as some guides online.

I'd also like to add that this is not "the" appropriate test, but one of many appropriate tests for the question you are asking. Another good—and at times mathematically equivalent—approach would be latent growth models.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.