2
$\begingroup$

There is no standard recommendation for length of errorbars to be used while showing spread of data or means in graphics. Standard deviation (SD), standard error of mean (SEM) and 95% confidence intervals (CI) are all used.

It is also common to have 2 means with errorbars shown in a single graph. An obvious question arises: whether non-overlap of errorbars indicate that the difference between 2 means is statistically significant (P<0.05)?

With reference discussions on this and this questions, it seems that following is usually accurate:

For 2 Means to be significantly different (P<0.05), the errorbars of length SEM*√2 should not be overlapping.

In contrast to above, errorbars of length 2*SEM (95% confidence interval) may be overlapping even if difference between 2 series is significant (P<0.05).

On the other hand, errorbars of length SEM may be non-overlapping even if difference between 2 series is not significant (P>0.05). Hence, SEM errorbars often give misleading impression of a significant difference between two similar series.

In view of above, why shouldn't errorbars of length SEM*√2 be used as a standard for graphical purposes?

Also, is there any specific name for this value: SEM*√2 ?

Improve this question
$\endgroup$
12
  • $\begingroup$ Considering your comment under the answer by @dariober, it is not clear waht your question is: Are you asking why error bars are drawn one SE in each direction and not tow SEs? $\endgroup$
    – cdalitz
    Nov 25, 2020 at 12:06
  • $\begingroup$ Since confidence intervals if computed correctly tend to be asymmetric, it is hard to get too interested in methods that assume symmetry of errors. $\endgroup$
    – Frank Harrell
    Nov 25, 2020 at 12:24
  • $\begingroup$ @cdalitz I am suggesting that error bar on each side should be SEM*√2 $\endgroup$
    – rnso
    Nov 25, 2020 at 12:38
  • 1
    $\begingroup$ @Frank One might think that, but when you consider that comparison of two CIs compares the upper limit of one to the lower limit of the other, the asymmetries tend to balance out. Thus, this kind of application is much more robust than one might initially imagine. $\endgroup$
    – whuber ♦
    Nov 25, 2020 at 17:30
  • 1
    $\begingroup$ @whuber that is extremely well written. I just don't favor having to manipulate individual confidence intervals for the purpose of making comparisons, and rather would like to see confidence intervals constructed for the specific comparison of interest. $\endgroup$
    – Frank Harrell
    Nov 30, 2020 at 12:03

1 Answer 1

Reset to default
1
$\begingroup$

why shouldn't errorbars of length SEM*√2 be used by default for graphical purposes?

Maybe what you are saying is related to the least significance difference? Crawley's R Book (pdf seems to be freely available here) has a nice description under the Analysis of Variance chapter, page 514 (t-test can be considered a special case of ANOVA, right?):

With standard errors we could be sure that the means were not significantly different when the bars did overlap. And with confidence intervals we can be sure that the means are significantly different when the bars do not overlap. But the alternative cases are not clear-cut for either type of bar. Can we somehow get the best of both worlds, so that the means are significantly different when the bars do not overlap, and the means are not significantly different when the bars do overlap?

The answer is yes, we can, if we use least significant difference (LSD) bars. Let us revisit the formula for Student’s t test:

t = (a difference) / (standard error of the difference)

We say that the difference is significant when t > 2 (by the rule of thumb, or t > qt(0.975,df) if we want to be more precise). We can rearrange this formula to find the smallest difference that we would regard as being significant. We can call this the least significant difference:

LSD = qt(0.975,df) × standard error of a difference ≈ 2 × sediff.

In general, though, I would keep in mind that CI and p-values express two different characteristics (all this in the perspective of comparing just two groups of course):

  • CI tells you how precisely the mean in this group has been estimated and it is therefore unrelated to the mean and precision of another group

  • pvalue is related to the difference between groups and therefore it must account for the precision of estimates in both groups jointly

(The Q&A you link have a more formal explanation)

So I think in this view it shouldn't be too surprising that CI can overlap and still give p < 0.05. Then depending on what you want to emphasize it may be more appropriate to show CI or LSD or else.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Very well explained. However, LSD is 2*sediff while errorbars are generally SE of means (not difference). Can we call SEM*√2 also as LSD? $\endgroup$
    – rnso
    Nov 25, 2020 at 11:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.