1
$\begingroup$

Can you give me a simple (but preferably common) example (or R/Python simulation) of what are practical consequences of wrong interpretation of frequentist confidence intervals? Especially when they are interpreted as Bayesian credible intervals (an interval within which an unobserved parameter value falls with a particular probability)?

$\endgroup$
6
  • 1
    $\begingroup$ In what sense would R or Python code constitute an "interpretation"? Several possibilities come to mind, so it would be good to have the question restated less ambiguously. $\endgroup$
    – whuber
    Commented Nov 18, 2021 at 19:24
  • $\begingroup$ @whuber Usually, simulation helps to avoid math details. It would be great to have a simple example with the least amount of math possible where I could say: Look, here's why you shouldn't interpret confidence intervals as credible ones. $\endgroup$ Commented Nov 18, 2021 at 20:15
  • $\begingroup$ I have seen some credible intervals as discontinuous (e.g., covering separate ranges of support in multi-modal posterior distributions), and cannot recall seeing any confidence interval doing that. $\endgroup$
    – Alexis
    Commented Nov 18, 2021 at 20:27
  • 1
    $\begingroup$ @Alexis Disconnected confidence regions arise naturally in some problems, such as root finding. I posted an example at stats.stackexchange.com/a/446205/919. $\endgroup$
    – whuber
    Commented Nov 18, 2021 at 22:13
  • 1
    $\begingroup$ I think the R/Python & simulation request diminishes the worth of this questions, which should simply be what are practical consequences of interpreting frequentist confidence intervals in the same way credible intervals are interpreted? $\endgroup$
    – Alexis
    Commented Nov 18, 2021 at 22:47

1 Answer 1

3
$\begingroup$

Example 1

In this question

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I gave an example of how confidence intervals and credible intervals are different.

difference between credible and confidence intervals

The two intervals draw boundaries in a different way.

  • The credible interval creates boundaries such that the interval contains the parameter x% of the time conditional on the observation $\bar{x}$. And this requires a prior probability distribution for the parameter in order to be computed.

  • The confidence interval creates boundaries such that the interval contains the parameter x% of the time conditional on the true parameter $\theta$.

Example 2

In this question

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

another example of this difference between conditioning on the observation versus the parameter is given.

If the model of the data is

$$X \sim N(\theta,1) \quad \text{where} \quad \theta \sim N(0,\tau^2)$$

then an image of multiple trials/experiments looks like:

Wasserman example

The behavior of confidence and credible intervals is different and the boundaries are different.

  • The boundaries of the confidence interval are made in such a way that they contain the x% of the points in the vertical direction (for each given $\theta$ you have x/2% of the points above and below the boundary).
  • The boundaries of the credible interval are made in such a way that they contain the x% of the points in the horizontal direction (for each given $\bar{x}$ you have x/2% of the poins above and below the boundary).

The result is that the performance of the intervals differs depending on the observed observation $x$ or depending on the true value of $\theta$.

Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

Wrap up

The last graph sums up the effect of the wrong interpretation of the confidence interval. In the right image, we have the red curve for the confidence interval and it would be wrong to assume the green curve.

The practical consequences (Do we lose money? Is our presentation of a theory wrong or not correct?) will depend on the practical situation.

$\endgroup$
1
  • $\begingroup$ +1. That regression graph example is swell, Sextus Empiricus. It also made me think of Deming regression, total least squares, etc.. $\endgroup$
    – Alexis
    Commented Nov 19, 2021 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.