Practical consequences of wrong interpretation of confidence intervals

Question

Can you give me a simple (but preferably common) example (or R/Python simulation) of what are practical consequences of wrong interpretation of frequentist confidence intervals? Especially when they are interpreted as Bayesian credible intervals (an interval within which an unobserved parameter value falls with a particular probability)?

Answer 1

Example 1

In this question

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I gave an example of how confidence intervals and credible intervals are different.

The two intervals draw boundaries in a different way.

• The credible interval creates boundaries such that the interval contains the parameter x% of the time conditional on the observation $\bar{x}$. And this requires a prior probability distribution for the parameter in order to be computed.

• The confidence interval creates boundaries such that the interval contains the parameter x% of the time conditional on the true parameter $\theta$.

Example 2

In this question

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

another example of this difference between conditioning on the observation versus the parameter is given.

If the model of the data is

$$X \sim N(\theta,1) \quad \text{where} \quad \theta \sim N(0,\tau^2)$$

then an image of multiple trials/experiments looks like:

The behavior of confidence and credible intervals is different and the boundaries are different.

• The boundaries of the confidence interval are made in such a way that they contain the x% of the points in the vertical direction (for each given $\theta$ you have x/2% of the points above and below the boundary).
• The boundaries of the credible interval are made in such a way that they contain the x% of the points in the horizontal direction (for each given $\bar{x}$ you have x/2% of the poins above and below the boundary).

The result is that the performance of the intervals differs depending on the observed observation $x$ or depending on the true value of $\theta$.

Wrap up

The last graph sums up the effect of the wrong interpretation of the confidence interval. In the right image, we have the red curve for the confidence interval and it would be wrong to assume the green curve.

The practical consequences (Do we lose money? Is our presentation of a theory wrong or not correct?) will depend on the practical situation.