For each of several subjects, I have mean reaction times for each of several stimuli. For each individual subject, I want to regress the RTs (as Y) against the stimuli (as X) and see if I get a slope that is significantly different from zero. For that, I used Matlab's fitlm function which gives me, for the X predictor, a coefficient estimate and a p-value. I then want to see if, at the group level, if those coefficients (slopes) are significantly different from zero, using a 1-sample t-test.

My questions: 1) For each subject, is it correct to look at the p-value as an indication of whether the slope is significantly different from zero, or should I look at a goodness-of-fit measure instead? I thought that, as in the case of correlation, the question of slope is more to do with the notion of effect size than that of statistical significance

2) Is it correct to say that those p-values represent within-subject variability while the p-value of the 1-sample t-test represents between-subjects variability? In other words, can the effect (significant linear dependence between Y and X) be present at the subject level but not necessarily at the group level?


  • 1
    $\begingroup$ Are the X stimuli different in an ordered way, as might befit penalized regression, or are they more nominal, as would suit an ANOVA? It sounds like you could use a multilevel model to separate within-subject random effects on reaction time from group-level effects of your stimuli, which I think (but I'm not sure that) you'd want to treat as fixed effects. About p values vs. effect sizes: the difference is mostly in emphasis on (A) inference regarding evidence against the null hypotheses in the population and (B) your sample's best estimates of differences (if any) among groups... $\endgroup$ – Nick Stauner Jul 30 '14 at 20:46
  • 1
    $\begingroup$ The stimuli are the integers from 0 to 9, so they are on a continuous scale, although I'm sure they could be treated as nominal as well (stimulus #1, stimulus #2, ..). The effect (called SNARC) is usually demonstrated by showing that the regression slope is non-zero both (and this is the confusing part) at the subject-level as well as at the group-level. I wanted to keep my question simple but the Y axis actually consists of the differences in reaction times, between those given with the right hand and those given with the left hand. $\endgroup$ – z8080 Jul 31 '14 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.