1
$\begingroup$

I'm seriously thinking of doing the following experiment in the field of attention research. In this particular area there are some factors of importance that appear to modify participants' response times and error rates depending on the unique combinations of the factors. Previous research looked at 2 or 3 factors combined, but never the 5 most important ones all at the same time.

For kicks I tried to design an experiment that would do this and ended up with a 2 x 2 x 2 x 3 x 4 design.The factors will all be categorical and the dependent variables most likely response time and error rate.

If I generate 20-ish trials per factor combination I'll need about 2000-odd trials per participant. This is quite a lot but I don't think it's unreasonable if spaced over a couple of days and presentations are randomized. I've done thousands of these in a day during the pilot phase of another similar study.

If I can find 10-15 people to torture for two or three days (taking fatigue into account) would this be feasible?

I know it would be an absolute nightmare to analyse the data using factorial ANOVA's and this might be the reason why people avoid doing this. Multilevel models would probably be a bit better but I don't know enough about them to know if this would be a bad idea.

Has anyone ever tried an experiment like this or analysed data from an experiment like this?. I've seen some 3 x 3 x 3 factorial designs in the filed, but never have I encountered anything this...uhm...enthusiastic.

Are there any statistical techniques that would be able to handle something like this?

Improve this question
$\endgroup$
3
  • $\begingroup$ Your problem likely won't be the analysis. You can generate binders full of statistical output. Your problem will be writing it all up, extracting the relevant factors and interpreting them. After spending a few months on this, you yourself will understand what's going on, but I don't think someone who reads your paper afterwards will have the faintest clue. It's painful enough to read a paper explaining and interpreting a three-way interaction. Don't make people slog through a five-way one. $\endgroup$
    – Stephan Kolassa
    Commented Feb 23, 2015 at 7:29
  • $\begingroup$ Thanks for the comment Stephan. I think you might have a point here. This will generate mountains of output and even a rudimentary reporting of the results will kill a forest or slow down the internet. I might end up doing this just to gain a better understanding and gain some insight into the basic problem and then generate some more targeted hypotheses from there. $\endgroup$
    – John Petrucci
    Commented Feb 23, 2015 at 7:54
  • $\begingroup$ Just be sure not to generate hypotheses by looking at some data and then use the same data to check those hypotheses. $\endgroup$
    – Stephan Kolassa
    Commented Feb 23, 2015 at 7:56

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.