4
$\begingroup$

When plotting within-subject data for condition A vs. condition B (significance tested through a t-test), should the error bars reflect standard error of the mean for each condition independently? Or should they be SE of the mean difference between A and B?

When there are more conditions for an ANOVA -- A vs. B for condition 1 and A vs. B for condition 2 -- should the error bars correspond to the SE of mean difference of A-B at 1 and then the SE of mean difference at 2?

Edit:

This paper gives a way to calculate SE/CI for within-subject designs. The gist is that the subjects are normalized to reduce the between-subject contribution to the error bars and better reflect the results of a repeated measures ANOVA.

Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorial in Quantitative Methods for Psychology, 1(1), 42–45. PDF

Improve this question
$\endgroup$
6
  • $\begingroup$ For clarity, is it the case that your experiment is 2 (before vs. after) X 2 (condition A vs. condition B), eg? $\endgroup$
    – gung - Reinstate Monica
    Jun 8, 2012 at 17:40
  • $\begingroup$ No, more similar to this: 2 (learned vs. novel) x 2 (face picture vs. house picture). Does it make a difference if there is explicit time information (before vs. after) tested? $\endgroup$
    – Amyunimus
    Jun 8, 2012 at 17:47
  • $\begingroup$ Not necessarily, I just typically think of within-subject studies as being longitudinal in nature. Nonetheless, you have 2 factors, 1 of which is within-subjects & the other is between-subjects, is that right? $\endgroup$
    – gung - Reinstate Monica
    Jun 8, 2012 at 17:53
  • $\begingroup$ Nope, each subject learns some face and house pics, and then is tested on both learned and novel face and house pics. No between-subject factors. $\endgroup$
    – Amyunimus
    Jun 8, 2012 at 17:56
  • $\begingroup$ Is this a memory experiment? If so, it may be better to use Signal Detection Theory rather than to have a learned vs. novel factor. This tutorial looks like it might be helpful in introducing SDT. $\endgroup$
    – gung - Reinstate Monica
    Jun 8, 2012 at 18:10

3 Answers 3

Reset to default
2
$\begingroup$

Regarding the first question, I think this depends on how you plot the data. If you are using a bar graph with the individual means side by side then I would add the error bars for the individual means. If the height of the bar chart represents the difference of the two means then use the standard error for the mean difference. As to the second question if I understand you correctly you are looking at mean differences on subsets of the data where condition 1 applies in one case and condition 2 in the other. Since this is what you want to show I would use the corresponding standard error (i.e. for condition 1 provide the standard error for the mean difference for the data where condition 1 applies and do it the same way for the mean difference when condition 2 applies.

Improve this answer
$\endgroup$
2
  • $\begingroup$ With two different SE of mean diff values for condition 1 and 2, should these be plotted on both bars within these conditions? The values would be identical... $\endgroup$
    – Amyunimus
    Jun 8, 2012 at 18:25
  • $\begingroup$ I would but I think in all cases this is really all a matter of choice as to what you think is the best display of what you want to show. $\endgroup$
    – Michael R. Chernick
    Jun 8, 2012 at 19:32
2
$\begingroup$

The standard error is generally an estimate of how well you've measured what you're interested in. In a repeated measures design you generally do not collect enough subjects (intentionally) to have really good estimates of the raw values. What you collect is enough subjects to have good estimates of effects. Therefore, not only would I use CI's (derived from SE's) of effects, but I would plot the effects themselves. Put the raw values and their standard deviations in a table. They are important but they aren't really about being persuasive or conveying the message of your story. Plots aren't about showing all of your data, tables are much more accessible for that. They're supposed to be about showing what's compelling about your data and making the point of your story more forcefully. Any error bar around the raw values in a plot is going to be in some way misleading inferentially when using a repeated measures design.

There have been some recent proposals to plot both kinds of error bars at the same time, either side by side, using colours or weights. This is possible as well. But I still prefer the plotting of effects with repeated measures designs. If you have a few measures then side by side graphs with the raw values without error bars (or maybe standard deviations) next to a plot of effects with confidence intervals of those effects is very nice.

Improve this answer
$\endgroup$
3
  • $\begingroup$ While I agree that it's important to have the raw values and SE/SDs accessible in table form, sometimes that's not an option -- when giving a talk or when summarizing results. In many cases, tables that are redundant with plots are not permitted in journal articles. CIs are good, but they obscure results if you have significant, but small effects. $\endgroup$
    – Amyunimus
    Jun 8, 2012 at 20:07
  • $\begingroup$ No, CI's of effects do NOT in any way obscure the results. The CI of the effect will either not cross 0 or will. It is very straightforward to interpret. And the SE?... that has no intpretation issues? Inference from it is dependant upon N and a distance beyond overlap of the bars. It's very rare to come across someone who can genuinely make a correct inference from an SE error bar. $\endgroup$
    – John
    Jun 8, 2012 at 21:15
  • $\begingroup$ Tables of raw values combined with graphs of effects are not redundant for repeated measures designs. You cannot accurately predict inferences about differences from the raw values and standard deviations, or standard errors, with repeated measures designs. I've never had a journal even comment on reporting both effects and raw values together. As for talks... you report what makes your story compelling, accurately. The graph I propose does just that with a report of the overall mean to place those effects. $\endgroup$
    – John
    Jun 8, 2012 at 21:25
1
$\begingroup$

The answer to your question depends on your objective: Is the purpose of this research to show differences between groups? If yes, then you should show the standard error of the difference. Make sure your figure caption is unambiguous, with, for example, an indication "The error bars show difference-adjusted standard errors".

In within-subject design, you can further take into account the correlation across participants to further narrow the error bars. These are then call "correlation- and difference-adjusted standard errors".

If the purpose is to show what scores were obtained in each condition, irrespective of what was found in the other conditions, then you need to use the basic (unadjusted) standard error. This last measure is also called stand-alone standard error because it is meaningful in isolation whereas the difference-adjusted are meaningful only when performing pair-wise comparisons.

In the social sciences, psychology and related fields, I believe that we commonly use repeated measures BECAUSE we want to compare scores. Hence, the difference-adjusted error bars are probably what you want most, if not all the time.

New readings here or here for R users.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.