Bayesian credible interval calculation: posterior distribution or marginal distribution Let's say we want to obtain the 95% credible interval for theta, where theta|x ~ Beta(a,b), which one of the following is correct and why?


*

*[E(theta|x) - 2*se(theta|x), E(theta|x) + 2*se(theta|x)]

*[E(theta)-2*se(theta), E(theta)+2*se(theta)]


Thanks ahead!
 A: In a Bayesian context, one does not use frequentist definitions of confidence intervals. That is, the notion of being a certain distance away from the mean times some amount of the standard deviation does not really make sense. Rather a 95% credible interval contains 95% of the value of the posterior distribution. Suppose we wanted a central 95% credible interval. Find theta such that the for the CDF, the lower interval point has at least 2.5% of data less than it and the upper interval has at least 97.5% below it (2.5% above).
Check out https://hilbertthm90.wordpress.com/2013/11/04/bayesian-statistics-worked-example-part-2/ for an example. Essentially the point is that when you are using your data to account for the randomness in your parameter theta, we can then assume that the posterior represents the mix of our views (prior) and the data (y). Hence we know that if we find the points such that 75% or 95% or 99% or any percent really lies underneath the probability mass function of the posterior, we can find the interval.
A: For a beta distribution, neither is going to be exactly right.
Given that the posterior has a closed form with density $p(\theta|x)$, a 95% credible interval is any interval $A$ such that $$\int_A \;p(\theta|x)\; d\theta = 0.95.$$
So, one could get the equal-tailed 95% credible interval using R with:
qbeta(c(0.025,0.975), a, b)

However, if the posterior is approximately normal, the first construction would approximately correct. Note that the posterior distribution conditions on the observed value of the data, $X=x$. Thus, in making posterior inference for $\theta$, we don't integrate over all possible values of $X$.
