Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite
1

My dependent variable are reaction times (RTs) in a psychological experiment. The experiment features a control condition and an experimental condition and for each subject, I have several hundred realizations of each condition (the variance in this experiment is pretty large, both the intra- and inter-individual). I calculate differences by subtracting the RTs in the experimental condition from those in the control condition and in the mean over trials and subjects, I get a value significantly different from zero.

Now my problem: From a theoretical perspective, I derive the hypothesis that the experimental condition differs only in some of the trials significantly from the control condition and that the differences in the means comes from this proportion, only. In other words, I suspect that the RT from some trials in the experimental condition is drawn from the same distribution as the control trials and only some are from a different distribution. I could possibly come up with a theoretical estimate of the size of this proportion but an approach that would allow to estimate it from the data would be even better.

Is there a way to statistically test this hypothesis against the hypothesis that all experimental trials are from the experimental distribution?

[EDIT]

I guess, I can rephrase the problem as follows: Can I test whether the data from the experimental condition is from a bimodal distribution rather than from a unimodal one and, furthermore, is one of the modes identical to the mode of the control-distribution?

[/EDIT]

thanks a lot,

matthias

share|improve this question
1  
Can you somehow identify the trials in which you expect the results to be the same (vs. being different)? E.g., if you expect that the difference is only due to the first $K$ trials, and then the subjects get trained enough in the conditions of the experiment that the difference wears out, you can devise some sort of cusum test to check whether the series indeed converge to the common mean, or look for a structural break in the individual time series. You would have to correct for multiple testing though. – StasK Sep 30 '11 at 18:30
I'm confused about the unit of analysis in this study: are you trying to identify the average treatment effect on each subject? the average treatment effect on the entire subject pool? something else? At the moment, it sounds like you're trying to identify both from the same data and the same test. – ashaw Sep 30 '11 at 19:33
@Stask: I did not mean such training effects, though this approach appears to me to be very interesting... Maybe we can discuss this in an own question, later. For now, let me assume that I do not know for each trial whether it contributes to the difference between control and experimental condition, or not. I was thinking about somethiing like an (in-)dependent mixture model for that but I'm not sure how such a thing would be fitted... – thias Sep 30 '11 at 20:59
@ashaw: Usually, I calculate a mean per subject both for the experimental and control condition and subject these to the test-statistics (ANOVA/t-tests, with repeated measures). That gives me the treatment effects. The question concerns individual RT's, though (or their distribution per subject). – thias Sep 30 '11 at 21:01
(1) Are you trying to figure out which trials would have a difference between conditions? If so your dependent variable is the categorization of trials rather than the mean RT. (2) Do you have a theoretical reason why trials would be different? If so, you might try some form of unsupervised classification. (3) If you answered "no" to both of those questions, I suspect that the issue is that you are using indicator variables for the different participants and just want to be able to interpret them, in which case you might try a multilevel model. I can answer further based on your response. – Thomas Levine Oct 4 '11 at 20:04
show 1 more comment

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.