6
$\begingroup$

Motivation: I was standing in front of a class to introduce into the concept of confidence interval using the example of differences in means (purely frequentist setting) and I was torturing the students with this inconvenient interpretation of confidence intervals. After a while, one student asked: "in the beginning of the lecture you told us that a confidence interval is more useful than a pure p-value because p-values are difficult to interpret and have a different meaning than what many people think they do - but your confidence interval seems to suffer from the same drawback". I immediately thought, "Well, he's got a valid point. Let's go Bayesian then!", but this would have definitely blown the limits of the course.

To cite from AdamO's answer in Interpretation of confidence interval:

The textbook definition of a $100 \times (1-\alpha)$% confidence interval is:

An interval which, under many independent replications of the study under ideal conditions, captures the replicated effect measurement $100 \times (1-\alpha)$% of the time.

Now, when we are in the Bayesian framework, we can construct so-called credible intervals, which allow us to state conclusions like:

[Given the prior distribution that we used,] for a 95% credible interval, the value of interest (e.g. size of treatment effect) lies with a 95% probability in the interval. (source)

Of course, to people who are not trained in frequentist thinking the interpretation of a credible interval is much more intuitive than that of a confidence interval.

I know that one important reason for this difference is that in frequentist framework we do not think of the parameter as a random variable but as a fixed value, while the confidence interval itself is random. However, in certain situations we can prove that the confidence interval calculated from both frameworks will coincide. These "certain situation" particularly cover the most common cases, i.e. the case of mean differences and linear regression with weakly informative prior distribtions.

Here comes my question: if I can show that Bayesian credible intervals and frequentist confidence intervals coincide why can't I interpret the frequentist confidence interval in the Bayesian way?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ They quite rarely coincide, I am afraid. $\endgroup$
    – Xi'an
    Commented May 29, 2019 at 8:56
  • 1
    $\begingroup$ "Rarely" in terms of the number of different parameters one might calculate a confidence interval for, I agree. But "rarely" in terms of number of applications within an introductory statistics course? If a person never leaves the area of t-test, linear regression and binomial test, are there many situation where confidence and credible interval do not coincide? $\endgroup$
    – LuckyPal
    Commented May 29, 2019 at 10:42

1 Answer 1

Reset to default
2
$\begingroup$

A non-mathematical answer:

There are a lot of procedures that lead to the same answer but have completely different underlying mechanisms or operations.

One simple example would be to compare the median with the mean. Both rely on different operations and both have quite different interpretations, but in a lot of cases the answer is exactly the same. However, this does not mean that you can use the interpretation of the median when you explicitly report the mean and vise versa.

The same goes for Confidence Intervals and Credible Intervals.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I like your example. Let's continue on that: if I have a formal prove that for my data mean and median conincide, for example I sampled it in R via rnorm(n=1e7), isn't a valid conclusion to say: "for this sample, the mean represents the value for which 50% of the data is smaller and the other 50% of the data is larger." ? $\endgroup$
    – LuckyPal
    Commented May 29, 2019 at 11:11
  • 2
    $\begingroup$ I think the word "represents" is problematic here because the mean does not represent that definition. At best, this would be very misleading for readers. More accurate would be to say "coincide": "For this sample, the mean coincides with the value for which 50% of the data is smaller and the other 50% of the data is larger.", which is the same as saying "in this sample, the mean coincides with the median". $\endgroup$
    – Fred
    Commented May 29, 2019 at 11:19
  • 2
    $\begingroup$ To add to Dan's comment, using the word "represents" here is especially problematic because you're asking the question in the context of a teaching environment. Your phrasing seems like a good way to quickly confuse your students, or to throw them off when they see a sample where the mean and the median don't coincide. The same goes for credible and confidence intervals. $\endgroup$
    – Accidental Statistician
    Commented May 29, 2019 at 11:21
  • $\begingroup$ Two good points, thank you! So, I could say "for the case of a normal distribution the interpretation of the confidence interval of the mean difference coincides with the Bayesian interpretation"? Of course, I see that this will confuse students who have never heard of prior distributions. But would it be wrong if I use the Bayesian interpretation the next time when I perform a t-test and report the 95%-confidence interval if I add the sentence "since it is proven that confidence interval and credible interval coincide for this case"? $\endgroup$
    – LuckyPal
    Commented May 29, 2019 at 11:33
  • 4
    $\begingroup$ This might be nitpicking, but here the interpretation is not coinciding, the outcome is. If you report a 95% confidence interval for which the values for the Credible Interval are the same and you want to use the interpretation of the Credible Interval, why not just report the Credible Interval? $\endgroup$
    – Fred
    Commented May 29, 2019 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.