2
$\begingroup$

Participants in a study all respond to stimuli under two conditions, and some difference in response time is expected between Condition 1 and Condition 2.

I have three continuous outcome variables:
- A participant's average response time in Condition 1
- A participant's average response time in Condition 2
- The difference between C1 and C2, i.e. C1 minus C2

I also have 7 variables that are believed to predict these outcome variables. In particular, they're strongly believed to predict the difference score.

Is it appropriate to use multiple regression to predict a difference score of this sort, i.e. to do three regression analyses, one for each outcome variable?

What differences in interpretation will be required as a result of the fact that I'm predicting a difference score?

Are there any other things that I will need to handle differently in this context, for example in relation to regression assumptions and diagnostics?

$\endgroup$

1 Answer 1

3
$\begingroup$

You will get more precision and power by analyzing all the raw data, not just averages and not just differences in averages. This can be done using generalized least squares, mixed effects models, and Bayesian hierarchical models.

If you proceed with the current plan make sure that the response times are perfectly transformed so that differences have the desired meaning. Make a Tukey mean-difference plot (Bland-Altman plot) and check that the difference is unrelated to the average times and that variability is constant across levels of the average.

$\endgroup$
5
  • $\begingroup$ You say "check that the difference is unrelated to the average times and that variability is constant across levels of the average". Should that check be done subjectively by looking at the Tukey mean-difference plot (Bland-Altman plot)? Or is there some other test I should do in addition to looking at the plot? $\endgroup$ Sep 8, 2015 at 13:45
  • 2
    $\begingroup$ You can run a Spearman correlation but this is usually done subjectively. $\endgroup$ Sep 8, 2015 at 14:07
  • $\begingroup$ I created a Bland-Altman plot [i.imgur.com/QvqXH48.png]. Subjectively it looks pretty good to me, but because I was playing with it before reading your response I ran a linear regression with the difference score C1-C2 as the outcome variable, and the mean score (C1+C2)/2 as the predictor. The predictor was marginally statistically significant, which I understand is undesirable. However, would you agree things look basically OK? $\endgroup$ Sep 8, 2015 at 14:16
  • 1
    $\begingroup$ The plot looks reasonable to me in terms of satisfying assumptions. Don't forget to consider analyzing the raw data though. $\endgroup$ Sep 8, 2015 at 15:07
  • $\begingroup$ Thanks for the advice. I made a separate question for that topic, as I believe it is separate from the current question: stats.stackexchange.com/questions/171853/…. $\endgroup$ Sep 10, 2015 at 8:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.