2
$\begingroup$

When plotting within-subject data for condition A vs. condition B (significance tested through a t-test), should the error bars reflect standard error of the mean for each condition independently? Or should they be SE of the mean difference between A and B?

When there are more conditions for an ANOVA -- A vs. B for condition 1 and A vs. B for condition 2 -- should the error bars correspond to the SE of mean difference of A-B at 1 and then the SE of mean difference at 2?

Edit:

This paper gives a way to calculate SE/CI for within-subject designs. The gist is that the subjects are normalized to reduce the between-subject contribution to the error bars and better reflect the results of a repeated measures ANOVA.

Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorial in Quantitative Methods for Psychology, 1(1), 42–45. PDF

$\endgroup$
  • $\begingroup$ For clarity, is it the case that your experiment is 2 (before vs. after) X 2 (condition A vs. condition B), eg? $\endgroup$ – gung Jun 8 '12 at 17:40
  • $\begingroup$ No, more similar to this: 2 (learned vs. novel) x 2 (face picture vs. house picture). Does it make a difference if there is explicit time information (before vs. after) tested? $\endgroup$ – Amyunimus Jun 8 '12 at 17:47
  • $\begingroup$ Not necessarily, I just typically think of within-subject studies as being longitudinal in nature. Nonetheless, you have 2 factors, 1 of which is within-subjects & the other is between-subjects, is that right? $\endgroup$ – gung Jun 8 '12 at 17:53
  • $\begingroup$ Nope, each subject learns some face and house pics, and then is tested on both learned and novel face and house pics. No between-subject factors. $\endgroup$ – Amyunimus Jun 8 '12 at 17:56
  • $\begingroup$ Is this a memory experiment? If so, it may be better to use Signal Detection Theory rather than to have a learned vs. novel factor. This tutorial looks like it might be helpful in introducing SDT. $\endgroup$ – gung Jun 8 '12 at 18:10
2
$\begingroup$

Regarding the first question, I think this depends on how you plot the data. If you are using a bar graph with the individual means side by side then I would add the error bars for the individual means. If the height of the bar chart represents the difference of the two means then use the standard error for the mean difference. As to the second question if I understand you correctly you are looking at mean differences on subsets of the data where condition 1 applies in one case and condition 2 in the other. Since this is what you want to show I would use the corresponding standard error (i.e. for condition 1 provide the standard error for the mean difference for the data where condition 1 applies and do it the same way for the mean difference when condition 2 applies.

$\endgroup$
  • $\begingroup$ With two different SE of mean diff values for condition 1 and 2, should these be plotted on both bars within these conditions? The values would be identical... $\endgroup$ – Amyunimus Jun 8 '12 at 18:25
  • $\begingroup$ I would but I think in all cases this is really all a matter of choice as to what you think is the best display of what you want to show. $\endgroup$ – Michael Chernick Jun 8 '12 at 19:32
2
$\begingroup$

The standard error is generally an estimate of how well you've measured what you're interested in. In a repeated measures design you generally do not collect enough subjects (intentionally) to have really good estimates of the raw values. What you collect is enough subjects to have good estimates of effects. Therefore, not only would I use CI's (derived from SE's) of effects, but I would plot the effects themselves. Put the raw values and their standard deviations in a table. They are important but they aren't really about being persuasive or conveying the message of your story. Plots aren't about showing all of your data, tables are much more accessible for that. They're supposed to be about showing what's compelling about your data and making the point of your story more forcefully. Any error bar around the raw values in a plot is going to be in some way misleading inferentially when using a repeated measures design.

There have been some recent proposals to plot both kinds of error bars at the same time, either side by side, using colours or weights. This is possible as well. But I still prefer the plotting of effects with repeated measures designs. If you have a few measures then side by side graphs with the raw values without error bars (or maybe standard deviations) next to a plot of effects with confidence intervals of those effects is very nice.

$\endgroup$
  • $\begingroup$ While I agree that it's important to have the raw values and SE/SDs accessible in table form, sometimes that's not an option -- when giving a talk or when summarizing results. In many cases, tables that are redundant with plots are not permitted in journal articles. CIs are good, but they obscure results if you have significant, but small effects. $\endgroup$ – Amyunimus Jun 8 '12 at 20:07
  • $\begingroup$ No, CI's of effects do NOT in any way obscure the results. The CI of the effect will either not cross 0 or will. It is very straightforward to interpret. And the SE?... that has no intpretation issues? Inference from it is dependant upon N and a distance beyond overlap of the bars. It's very rare to come across someone who can genuinely make a correct inference from an SE error bar. $\endgroup$ – John Jun 8 '12 at 21:15
  • $\begingroup$ Tables of raw values combined with graphs of effects are not redundant for repeated measures designs. You cannot accurately predict inferences about differences from the raw values and standard deviations, or standard errors, with repeated measures designs. I've never had a journal even comment on reporting both effects and raw values together. As for talks... you report what makes your story compelling, accurately. The graph I propose does just that with a report of the overall mean to place those effects. $\endgroup$ – John Jun 8 '12 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.