Multilevel modeling of response time data

Question

I'm trying to figure out how to set up and analyze the following experiment.

It's a basic reaction time-type experiment with 4 independent variables (2 levels each) and 1 dependent variable (RT).

The 4 independent variables are:

1. Hand used for response (Right vs Left)
2. Side of visual field where target appeared (left vs right)
3. Exposure Time (400ms vs 800ms)
4. Priming of target (Primed vs Not-primed)

Basically I want to see how these things interact in order to test a specific hypothesis about their interaction.

The problem with this experiment is the fact that I will be generating about 30 observations per participant for each unique combination of those variables. From what I can gather repeated-measures ANOVA would not be the way to go as this would require reducing those multiple observations to a mean, losing valuable data.

Some posts here (and elsewhere) suggest multilevel modeling for analyzing this type of experiment. What would be the most efficient way of setting up this experiment and analyzing the data? Should I look at grouping the variables a certain way or can I just chuck the data into long format, use the MIXED function in SPSS and call it a day? From the pilot testing it appears as if the distributions will be positively skewed as with most RT data of this nature. Should i be performing some log-like transformation before doing the analyses?

p.s Is this thing even called a 2x2x2x2 factorial design?

Answer 1

That's a lot of questions rolled into one but it looks like you have done some background research so I'll proceed.

Yes, this would be such a design as you describe.

You don't use multi-level modelling in order to retain all of the data. You use it because it's a more correct model of what your data represent. And yes, you use every single score, no aggregating or combining things.

The multi-level model will be more sensitive to the skew issue where the central limit theorem might have corrected for it in the ANOVA. You don't transform the data first. Run the model and then check the residuals for skew. You could run a Box-Cox test on them. As a rule of thumb, if lambda comes close to 0 then you can use a log transform, if it's close to -1 then use an inverse transform. Then run your model and check the outcome.

About RTs, in your task the log will probably be a slightly better transform to generate normal residuals. But an inverse transform on the RT in seconds may be the best way to go in general because it can often bring you very close to normal and is easily interpreted. Some people are very wedded to RT expressed in seconds or milliseconds as the measure but it's actually pretty arbitrary how the performance is represented. The inverse is simply the rate or number of responses that could be made per second. So, 4 is the inverse of 250ms. It's hard to argue one of those is the only correct way to represent speed of responding.