9
$\begingroup$

Given a confidence interval with 95% confidence level, it is incorrect to state that the probability of the estimated parameter being included in the confidence interval is 95%.

The reason is that the sample has already been taken; the real value either is included in it, or not. The notion of randomness no longer applies.

But my question is what is the importance of this distinction?

P.S. If you can illustrate the answer with a down to earth example, that a person without a strong background to statistics can understand, I'd appreciate it.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

I'll be honest there: I don't think the actual distinction is all that important. Yes, saying that "the probability of the estimated parameter being included in the confidence interval is 95%" is incorrect, for the precise reason you give. However, I do not think it is a major problem. (I would be interested in any other point of view. Has this incorrect manner of writing ever led to "real" problems?)

If you run a single experiment and get a single CI, then yes, it either contains or does not contain the true value of the parameter:

single CI

As you write, there is no probability involved any more. The correct interpretation of a CI only comes in if we (explicitly or implicitly) run precisely the same experiment many times and collect all CIs:

100 CIs

And here, we see that (approximately) 95% of the CIs do contain the correct parameter. (The CI from the single experiment pictured above is the one at the bottom in this second plot.)

Yes, it would be better if everyone used the correct nomenclature, or at least had the correct interpretation involving many re-runs of the experiment in the back of their head while they were writing sloppily. But people don't.

And I honestly don't think this is a truly big deal.

R code:

set.seed(1)
n_population <- 1e6
xx_population <- runif(n_population)
param <- 0.5
yy_population <- 2+param*xx_population + rnorm(n_population, 0, 0.5)

n_analyses <- 100
n_sample <- 30

CIs <- matrix(NA, nrow=n_analyses, ncol=3)
for ( ii in 1:n_analyses ) {
    index <- sample(1:n_population,n_sample)
    model <- lm(yy_population[index]~xx_population[index])
    CIs[ii,] <- c(confint(model)[2,1],coef(model)[2],confint(model)[2,2])
}

opar <- par(mai=c(.5,.1,.1,.1))
ii <- 1
plot(range(CIs), c(ii,ii), type="n", xlab="", ylab="", yaxt="n")
lines(CIs[ii,c(1,3)], rep(ii,2), 
      col=2-(CIs[ii,1] < param&param < CIs[ii,3]))
points(CIs[ii,2], ii, pch=19, col=2-(CIs[ii,1]<0.5&0.5<CIs[ii,3]))
abline(v=param,lty=2,lwd=2)

plot(range(CIs),c(1,n_analyses),type="n",xlab="",ylab="",yaxt="n")
sapply(1:n_analyses,function(ii)lines(CIs[ii,c(1,3)],rep(ii,2),
        col=2-(CIs[ii,1]<param&param<CIs[ii,3])))
points(CIs[,2],1:n_analyses,pch=19,col=2-(CIs[,1]<0.5&0.5<CIs[,3]))
abline(v=param,lty=2,lwd=2)
Improve this answer
$\endgroup$
7
  • $\begingroup$ Grear answer! Very informative! $\endgroup$
    – stats_noob
    Commented Dec 24, 2022 at 22:54
  • $\begingroup$ @ Stephan Kolassa: I re-read your answer again, and here is my take away: If you collect some data, fit a regression model to this data and estimate regression coefficients (e.g. b1) and then calculate the 95% confidence interval for b1 (e.g. between some range x,y) - it is incorrect to say "there is a 0.95 probability that this confidence interval contains the true value of b1. However, if you could collect data many times, 95% of these times the original interval (x,y) would contain the true value of b1 (i.e. b1 from the population). Is this correct? $\endgroup$
    – stats_noob
    Commented Dec 25, 2022 at 2:52
  • $\begingroup$ @stats_noob: yes indeed. (Always assuming a correctly specified model, of course, which is already a quite strong assumption. You could try the simulation above with a misspecified model and see how often the CI contains the true parameter.) $\endgroup$
    – Stephan Kolassa
    Commented Dec 25, 2022 at 6:12
  • $\begingroup$ @ Stephan Kolassa: This point has always confused me so much. I could just turn the tables and say - that is a 0.95 probability that the data I collected and the confidence interval calculated from this data that I collected is one of those experiments in which the confidence interval bounds the true value of the parameter. Furthermore, in this amazing R visualization you provided - many of these confidence intervals contain the true value of the parameter AND many of these confidence intervals that contain the true value of the parameter have significant overlap with each other. $\endgroup$
    – stats_noob
    Commented Dec 25, 2022 at 6:42
  • 1
    $\begingroup$ I spent the whole day reading about this topic and ended up posting a related question over here (stats.stackexchange.com/questions/600000/…) in which I provide a link to your amazing R data visualization! $\endgroup$
    – stats_noob
    Commented Dec 25, 2022 at 7:12
0
$\begingroup$

Suppose you are estimating the price of a house using some given features. So when someone says to find out the $95$% confidence interval of the price, given some features. So you are basically finding an interval $[x, y]$ so that the probability of your price lying in this interval $[x,y]$ is $0.95$ (i.e $95$%).

I hope this helps.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.