3
$\begingroup$

I have noticed that a lot of literature usually refers to the frequentist confidence interval as "exact" in comparison to the Bayesian probability/credibility interval, which is calculated off the posterior distribution. In this sense, are Bayesian probability intervals viewed as an approximation of the frequentist interval? This doesn't make intuitive sense as the Bayesian and Frequentist intervals seem to be inherently different. Making things more confusing, I know that sometimes the Bayesian and Frequentist intervals will match up:

For example, for a case where the 95% probability interval is also a 95% "exact" confidence interval is the case where we assume $X_1, \ldots, X_n \sim \mathcal{N}(\mu, \sigma_1^2)$ are iid, and that $Y_1, \ldots, Y_n \sim \mathcal{N}(\mu, \sigma_2^2)$, the marginal posterior distribution for $\mu$ has a 95% probability interval (bayesian) that is also the same as the "exact" 95% confidence interval (frequentist).

Would anyone be able to help me see where my understanding is wrong or confused? Thanks!

$\endgroup$
  • 3
    $\begingroup$ Doesn't "exact" in this case refer to confidence intervals built off of exact tests (en.wikipedia.org/wiki/Exact_test)? Given that Bayesian inference is usually not concerned with rejecting null hypotheses, such terminology wouldn't be applicable. $\endgroup$ – C.R. Peterson Dec 9 '15 at 11:27
  • $\begingroup$ I would not put too much value in this denomination of "exact": a 96% confidence interval is exact in the sense that it contains the true value of the parameter for 96% of the samples. While the Bayesian 96% credible interval contains the parameter with probability 0.96 under the posterior, for a given realisation of the sample. They match up in some rare cases and almost always asymptotically in the sample size $n$. $\endgroup$ – Xi'an Dec 9 '15 at 20:47
2
$\begingroup$

Frequentist confidence intervals can be exact, asymptotically exact, or not exact. If they are exact the 96% confidence interval has a frequentist coverage of 96%. If they are asymptotically exact, there exist a limit - usually a large sample - at which the confidence interval becomes exact. If they are not exact, they have no guaranteed frequentist coverage, and are as such of limited use for frequentist inference.

In practice in frequentist statistics, we may have complex models with no known exact test, and limited data far away from any asymptotics. In such a situation, we do not really know what we should do to get our frequentist confidence intervals. One option is to use asymptotically valid tests and not to check the validity of the asymptotics. The second option involves Bayesian posterior probability intervals.

In Bayesian statistics, it is nowadays possible to determine Bayesian posterior probability intervals for complex models with limited data, provided one has sufficient computational resources. Further for some problems with a suitable symmetry of the space of observations and the space of parameters, Bayesian posterior probability intervals with “non-informative” priors are identical to the frequentist confidence intervals, even though the two answer different questions.

Thus the second option to get frequentist confidence intervals if you do not have an exact test and limited data far away from any asymptotics: use your posterior probability intervals as frequentist confidence interval, and hope that you are close to a situation where the two correspond.

The explanation above has aspects that are more an opinion than an answer. Please comment, if you disagree. Also I think that it would be better to have frequentist exact test that can be evaluated for complex models with limited data. I made some attempts in this direction https://dx.doi.org/10.6084/m9.figshare.1528163 . This needs however some more work.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.