True parameter in relation to credible interval

Question

I know that in the frequentist approach, the confidence interval contains the true parameter $\theta$ with some minimum probability (e.g. 95%); while in the bayesian approach, the credible interval includes 95% of the posterior distribution.

My question is, how would we relate the credible interval (with left and right bounds L and R) to the true parameter? In other words, how would we find $P(L(X) \leq \theta \leq R(X)$?

To put my question in another way: given a set of data, we can derive some posterior distribution (and thus credible interval), but that doesn't guarantee the true parameter is captured by this credible interval. How would I find the probability that the true parameter lies within the credible interval?

Answer 1

Bayesian credible interval. In Bayesian estimation of binomial success probability $\theta,$ suppose we begin with the prior distribution $\theta \sim \mathsf{Beta}(2. 2).$

Upon observing $x = 45$ successes in $n = 100$ trials, we have likelihood proportional to $\theta^{45}(1-\theta)^{55}.$

Then according to Bayes' Theorem, the posterior distribution is $\theta \sim \mathsf{Beta}(47. 57),$ and 95% Bayesian posterior probability interval (credible interval) $(0.358, 0.548).$

qbeta(c(.025, .975), 47, 57) 
[1] 0.3578055 0.5477928

If we believe the prior and the integrity of the data, then be must believe the posterior probability distribution is correct. According to the posterior distribution, $P(0.358 < \theta < 0.548) = 0.95.$

I'm puzzled by your sentence, "[T]hat doesn't guarantee the true parameter is captured by this credible interval." There is no absolute "guarantee" attached to any interval estimate [other than $(0,1)$]. However, under the posterior probability distribution, there is probability 95% that the credible interval covers $\theta.$

---

Agresti-Cooll frequntist confidence interval. For the same data as above we have a 95% A-C confidence interval %(0.356, 0.548).$

p = 47/104; p+qnorm(c(.025,.975))*sqrt(p*(1-p)/104)
[1] 0.3562732 0.5475730

One frequentist interpretation is that the process by which this CI was obtained would contain the true unknown value of $\theta$ over the long run in future repetitions of the experiment that produced the data.