4
$\begingroup$

I have heard this advice repeatedly however recently when I was looking at my own graph with CIs I had a panic attack because the error bars overlapped, yet my analysis told me the difference between the means was significant. I later learned here that I was making an incorrect assumption that the CIs couldn't overlap. I don't know how idiosyncratic my error was to me, or if there's any reason to suspect those who see my bar graph might draw the same erroneous conclusions. Do you recommend one over the other? Are CIs really less prone to misinterpretation than SEs?

$\endgroup$
  • $\begingroup$ Where have you heard that advice? There's certainly sense in it when the distribution of the mean in question doesn't follow an approximately normal distribution. Some context would make this q. clearer. $\endgroup$ – Scortchi Jun 6 '16 at 19:44
  • 1
    $\begingroup$ Reporting CIs instead of SEs is increasingly preached in my field (psychology) and beyond as the right thing to do. It is said that researchers use SEs because they are smaller (so it looks like they are trying to mask imprecision in their estimates) and CIs add more information than point estimates. I'm skeptical, however, most researcher will understand how to interpret them and will not make the mistake I did. I have a graph now which has CIs for two means (that are significantly different) that overlap a fair amount and I'm wondering if it's displaying CIs really is such a good idea. $\endgroup$ – PanPsych Jun 10 '16 at 14:45
  • 3
    $\begingroup$ Here's a cynic's rule of thumb: use SE's when you need to emphasize apparent differences and CI's when you wish to de-emphasize them. :-) $\endgroup$ – whuber Jun 10 '16 at 22:37
2
$\begingroup$

I don't know whether standard errors or confidence intervals are more liable to misinterpretation & suspect there's not much in it. If pairwise differences in parameter estimates are of particular interest you should report them together with their SEs/CIs, & thus forestall readers' drawing wrong conclusions from overlapping, or non-overlapping, SEs/CIs of individual parameter estimates.

Reporting CIs is usually preferable for estimates whose sampling distribution is highly skewed: reporting SEs is rather an invitation to imagine a corresponding (symmetric) normal confidence distribution around the point estimate; & the intervals implied, as well as having incorrect coverage, will do a poor job of separating parameter values better supported by the observed data from those worse supported. (When the sampling distribution is not skewed, but otherwise not well approximated by the normal, e.g. a Student's t distribution with few degrees of freedom, incorrect coverage is usually the only concern.)

$\endgroup$
  • $\begingroup$ Can you explain why CIs are preferable when the sample distribution is skewed/non-normal? I hadn't heard this previously. My estimates are logits from a logistic regression analysis (outcome variable takes on only one of two values). $\endgroup$ – PanPsych Jun 11 '16 at 5:17
  • $\begingroup$ @panpsych your confidence bands are symmetric, same as always $\endgroup$ – Repmat Jun 11 '16 at 19:45
  • $\begingroup$ @Repmat: Well, that depends how you calculate them! With small sample sizes or extreme probability estimates it's often a good idea to calculate the asymmetric confidence intervals got from the profile likelihood. $\endgroup$ – Scortchi Jun 11 '16 at 19:48
  • $\begingroup$ @scortchi yeah sure I concur. But that was not what the comment asked $\endgroup$ – Repmat Jun 11 '16 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.