11
$\begingroup$

After stumbling upon the concept in a statistics textbook, I tried to wrap my head about it, and finally came to a conclusion which seems to fit all the explanations which I have seen so far: A credible interval is what non-statisticians think a confidence interval is.


Digression for those like me-from-an-hour-ago who don't know the difference

If we observed data and predicted some parameter from it, let's say the mean $\mu$, the credible interval is the interval $[\mu_{\text{min}},\ \mu_{\text{max}}]$ for which we are 95% sure that mu falls inside (or some number other than 95%, if we used another level). The confidence interval taught in introductory statistics classes can overlap with the credible interval, but will not always overlap well. If you want to brave the explanation, try reading this and this question on Cross Validated; what helped me finally understand, after much head-scratching, was this answer.


Does it mean that it would be scientifically preferable to use a credible interval over a confidence interval in my results? If yes, why haven't I seen any publications which use it?

  • Is it because the concept should be used, but the measuring scientists have not yet caught up with the correct statistic methods?
  • Or is the meaning of the original confidence interval better suited to explaining results from empirical studies?
  • Or is it that in practice, they are so often overlapping that it doesn't matter at all?
  • Does the choice depend on the statistical distribution we are assuming for our data? Maybe with a Gaussian distribution, they always overlap numerically, so nobody outside of pure statistic cares about the difference (many studies I have read don't even bother to calculate any kind of interval, and maybe around 1% ever give space to the thought that their data might not be normally distributed).
  • Does it depend on our scientific-theory standing? For example, it feels like the confidence interval should be used in positivist work and the credible interval in interpretivist work, but I am not sure that this feeling is correct.
$\endgroup$
4
  • $\begingroup$ Confidence intervals are for frequentist and credible intervals for Bayesian approach. "why haven't I seen any publications which use it?" there are plenty (Bayesian) $\endgroup$ Feb 6, 2014 at 3:47
  • 6
    $\begingroup$ As of today there are 154 articles on PubMed mentioning credibility interval and 489 mentioning credible interval. They are not as common as confidence interval (179811 articles and counting), but it's just due to frequentist approach being the dominating method. And yes, credible interval sounds awesome but it's only true if the prior distribution is correctly specified. The devils are all in the assumptions. $\endgroup$ Feb 6, 2014 at 13:39
  • $\begingroup$ I may have still have my terms mixed up, but in my textbook, the author is suggesting using a credible interval when estimating the mean of binomial data using a maximum likelihood estimation based on a test statistic derived from standard errors. And I think this is a frequentist approach. Is there maybe a difference between a credible interval and an "actual coverage probability" confidence interval? $\endgroup$
    – rumtscho
    Feb 6, 2014 at 16:22
  • 1

1 Answer 1

4
$\begingroup$

The type of interval indicates what type of method you used. If a credible interval (or Bayesian variant), it means a Bayesian method was used. If a confidence interval, then a frequentist method was used.

Re: Or is it that in practice, they are so often overlapping that it doesn't matter at all? As long as

  • the conditions to use methods are reasonably satisfied (e.g. "independence of observations" is a requirement for many methods),
  • the Bayesian method doesn't use an informative prior,
  • the sample that isn't very small, and
  • the models / methods are analogous,

the credible and confidence intervals will be close to each other. The reason: the likelihood will dominate the Bayesian prior, and the likelihood is what is typically used in frequentist methods.

I would suggest not fretting about which to use. If you want an informative prior, then be sure to use a Bayesian method. If not, then choose a suitable method and context (frequentist or Bayesian), check to make sure the conditions required to apply the method are reasonably satisfied (so important but so rarely done!), and then move forward if the method is appropriate for the type of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.