When subject-based analysis is better than answer-based (and vice versa)?

Question

In experimental psychology one of the generally accepted practices in data analysis is the aggregation on subject level. For example, several measurements of reaction time is gathered for each subject using different stimuli, then mean reaction time is computed for each subject. Those scores are further analyzed by ANOVA or other methods.

My questions are:

1. When this procedure is more appropriate than a simple analysis of raw data (i.e. the analysis without any aggregation)? And when analysis of raw data is more appropriate?
2. Has anyone met the discussion of these question in literature?

UPD: To give an example, suppose I'm interested in the effect of a drug on a speed of face detection. So I for each participant I have the data for drug and placebo condition (within subject design). I can analyze a) answers for each face for each subject for each condition, b) answers for each subject aggregated by condition, and, perhaps c) answers for each face aggregated by condition. When (a) is better than (b) and when it is otherwise?

Answer 1

Both, by-item and by-participant analysis are flawed. This is also known as the language-as-fixed-effect-fallacy. The generally promoted approach is to use mixed-modeling with crossed random effects for both subjects and items. The most recent article discussing this is:

Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54–69. doi:10.1037/a0028347

More references can be found in my answer to this question.

Note that you need a certain number of replications for each stimulus-subject pair to run a decent mixed model.

Answer 2

What do you mean by "simple analysis of raw data"?

I would say, given what you've written, that item-based analysis is more appropriate when you want to know about items, and subject-based is more appropriate when you want to know about subjects.

E.g, item analysis is often used in psychometrics, when constructing a test. One wants to know such things as how hard the item is, whether the people who got it right also got other questions right, whether there are patterns in the wrong answers, whether it correlates with other items etc.