0
$\begingroup$

I've made a bar plot and I calculated confidence intervals by hand,

(+/- 1.96*std_error)+predicted_probability

and I want to be sure that it's interpretable. That is, do they have the same interpretation that they would if the point estimate was a mean or a logit? I understand that CIs for odds ratios and probabilities are inherently non-symmetric, but does this affect their interpretation?

For example, if I had two means and 83% of the total length of their CIs were non-overlapping, then the means are significantly different at .05. Would the same be true of CIs for predicted probabilities?

$\endgroup$
  • 1
    $\begingroup$ "if I had two means and 83% of the total length of their CIs were non-overlapping, then the means are significantly different at .05" - not true. I think you mean that if you had estimates of two different means with the same standard error, non-overlap of the usual normal-theory CIs with 83% coverage around each would imply a p-value of less than 0.05 for the test for difference of the means. $\endgroup$ – Scortchi Jun 12 '16 at 10:39
  • $\begingroup$ You say you know the CI should be asymmetric but the formula you are using gives rise to a symmetric interval. It would be better to use the Wilson method to get the CI. $\endgroup$ – mdewey Jun 12 '16 at 12:58
  • $\begingroup$ @mdewey, I don't follow. When using this formula with log odds and probabilities the results are asymmetric, which is not surprising. My question is whether this means reporting these CIs is problematic. Can you explain why the Wilson method is preferable? $\endgroup$ – PanPsych Jun 12 '16 at 14:54
  • $\begingroup$ If you use your formula directly with probabilities (which was your final question) then the interval will be symmetric. If you have transformed the probabilities in some way you did not explain and then back-transformed the limits of the interval then fine. $\endgroup$ – mdewey Jun 12 '16 at 14:56
  • $\begingroup$ @Scortchi, I probably didn't explain that well but here it was explained to me: stats.stackexchange.com/questions/203744/… But my main question is whether what is true for means would hold for predicted probabilities (from a logistic regression)? $\endgroup$ – PanPsych Jun 12 '16 at 14:59
1
$\begingroup$

The issue of how to generate appropriate confidence intervals for a single proportion has been addressed repeatedly in the literature. A good example is

@ARTICLE{newcombe98a,
  author = {Newcombe, R G},
  year = 1998,
  title = {Two-sided confidence intervals for the single proportion:
          comparison of seven methods},
  journal = {Statistics in Medicine},
  volume = 17,
  pages = {857--872}}

the first of a series of three papers in which he also discusses confidence intervals for differences. He restricts himself to methods which do nto involve iteration and compares amongst other the Wald method (which is what the OP suggests), Wilson's score based method and the Clopper-Pearson method. He concludes that the Wilson method has good properties.

As for the question of what they mean the answer is that they are confidence intervals and mean the same as any other confidence intervals: in the long run they will contain the true value with the specified coverage.

$\endgroup$
  • $\begingroup$ Thank you! I'd like to use Wilson's method. Any suggestions about how to do this in R using estimates generated from glmer? Or is it simple enough to do by hand? I'm actively looking into this but if you know, that would be great. I might have to open another question. $\endgroup$ – PanPsych Jun 12 '16 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.