Converting a confidence interval into a credible interval

Question

The problem of correctly interpreting confidence intervals has been discussed at length here. I have a related question which I believe may be a useful contribution: Frequentist probabilities by definition refer to an infinite number of repetitions of an experiment. Therefore, the frequentist definition of a confidence interval (CI) is: If you extract an inifinite number of samples from a normal distribution with unknown parameters ($\mu,\sigma$) and calculate a CI from each sample via a defined algorithm $A_{95}$, said CI will contain $\mu$ in $95 \% $ of all cases. From a frequentist point of view, it doesn't make sense to apply that probability to a given CI to estimate its likelihood of containing $\mu$.

But what if someone were to switch from a frequentist to a Bayesian viewpoint after constructing the CI and ask themselves the question: "How confident am I (i. e. at what rate would I - given that there is someone who knows $\mu$ and will reveal it at some point - be willing to bet) that this given CI contains $\mu$ knowing that it was constructed using $A_{95}$?"

Wouldn't the answer be "$95 \%$" (or $20:1$)?

C. f.: Should I have "Confidence" in Credibility Intervals?

Answer 1

For so called location models, such as your linear regression, anovas etc., basically for models where the outcome depends linearly on the estimated parameters, the confidence interval will be the same as the credible interval with flat prior.

If you want to know how would that credible interval look like with a different prior, then you add that prior information to your results as you would do normally as a Bayesian.

This is not the case for models where the outcome depends non-linearly on parameters, such as for logistic regression. I think this is because likelihood is invariant to reparametrizations, but priors are not, so you cannot really have a non-informative prior in this case. There are also invariant priors, and probability matching priors, but unfortunately I don't know how they fit into the picture. Sorry

How much will confidence and credible intervals differ, depends on the data. The more data you have, the less will they differ. With a lot of data (infinite) they will be the same again. If they are different, credible intervals won't have the proper coverage, which might not be something you want.

If you want a more mathy treatment of this topic look at

Fraser, D. A. S. “Is Bayes Posterior Just Quick and Dirty Confidence?” Statistical Science, vol. 26, no. 3, 2011, pp. 299–316. JSTOR, www.jstor.org/stable/23059129. Accessed 3 June 2021. (there is also an arxiv version https://arxiv.org/abs/1112.5582)

There you will also find a formula to how much will they differ and also how to get from a confidence quantile to Bayesian quantile. The math in the paper should be good, but the sentiment of the paper is not something that is generally accepted.

Answer 2

"But what if someone were to switch from a frequentist to a Bayesian viewpoint after constructing the CI and ask themselves the question: "How confident am I (i. e. at what rate would I - given that there is someone who knows μ and will reveal it at some point - be willing to bet) that this given CI contains μ knowing that it was constructed using A95?""

Yes, that is often exactly what people do interpret confidence intervals without realising they are silently moving from one probabilistic framework to another, without stating the assumption that bridges the two.

It is often fairly benign as there may be a reasonable (i.e. non-contrived) Baysian prior for which the frequentist confidence interval and credible interval are numerically the same. From a subjectivist perspective, there is nothing wrong with viewing the long history of benign interpretations of confidence intervals as credible intervals as justifying that Bayesian confidence in credible intervals.

However the point is that they are answers to different questions so we should not expect the answer to be the same. Bayesians can also form confidence intervals if they choose to do so, I suspect they rarely do because it generally isn't the question you want to ask, and frequentists only use them because they can't give a direct probabilistic answer to the question you actually do want to ask.