1
$\begingroup$

In the book "all of statistics" the author brings the following interpretation of confidence intervals with the following exmaple: Interpretation: "On day 1, you collect data and construct a 95 percent confidence interval for a parameter $\theta_1$. On day 2, you collect new data and construct a 95 percent confidence interval for an unrelated parameter $\theta_2$ . On day 3, you collect new data and construct a 95 percent confidence interval for an unrelated parameter $\theta_3$ . You continue this way constructing confidence intervals for a sequence of unrelated parameters $\theta_1$ ,$\theta_2$ , . . . Then 95 percent of your intervals will trap the true parameter value. There is no need to introduce the idea of repeating the same experiment over and over." Example: "Every day, newspapers report opinion polls. For example, they might say that "83 percent of the population favour arming pilots with guns." Usually, you will see a statement like "this poll is accurate to within 4 points.95 percent of the time." They are saying that 83 ± 4 is a 95 percent confidence interval for the true but unknown proportion p of people who favour arming pilots with guns. If you form a confidence interval this way every day for the rest of your life, 95 percent of your intervals will contain the true parameter. This is true even though you are estimating a different quantity (a different poll question) every day."

can someone explain why this is true? Why can we know 95 percent of the intervals contains the true parameter even though we are estimating a different quantity?

Improve this question
$\endgroup$
3
  • $\begingroup$ By construction: each CI is supposed to be based on independent data in this scenario and to cover the true parameter with 95% probability. (Neither of those is ever truly the case, but that's what the author implicitly supposes.) The Weak Law of Large Numbers answers your question. $\endgroup$
    – whuber ♦
    Commented Feb 13, 2022 at 19:02
  • $\begingroup$ @whuber I don't understand why if I have the result of a poll on 10 people for president1 on day 1 and then another poll on hamburgers day 2 and then another poll on the number of children on day 3, and so on, why can I use these independent data to get the confidence interval for a new poll on the popularity of movie1? How is this possible? The author says it's true even when we are focusing on a different poll question in each day $\endgroup$
    – Mina
    Commented Feb 13, 2022 at 20:50
  • $\begingroup$ The quotation doesn't assert you can use old, independent data to construct confidence intervals for new data. It is only making a statement about the long-term rate at which you, as a user of 95% CIs, can expect them to cover their true values, even though in any particular case you will not know whether the value is covered. $\endgroup$
    – whuber ♦
    Commented Feb 13, 2022 at 21:26

1 Answer 1

Reset to default
3
$\begingroup$

Of course we need to assume that the model assumptions always hold.

Now consider different $\beta$-level (say $\beta=0.95$) confidence intervals $C_1,\ldots,C_n$ for parameters $\theta_1,\ldots,\theta_n$. I also assume that the confidence level is precise for all intervals, i.e., $P\{\theta_i\in C_i\}=\beta,\ i=1,\ldots,n$, and that data for all the different intervals are independent.

I can then define binary (0-1) random variables $I_1,\ldots,I_n$ so that $I_i=1$ if $\theta_i\in C_i$, i.e., the confidence interval $i$ catches true parameter $\theta_i$.

This defines an independent sequence of Bernoulli-experiments with probability $\beta$ for observing 1. The relative frequency of successes of the confidence interval then behaves as the standard estimator for a probability $p$ (here equal to $\beta$) in such an experiment, which is the relative frequency of observing 1 (equal to the arithmetic mean of the zeroes and ones). The Laws of Large Numbers then state that this will converge (in probabilistic terms) toward $\beta$ (it will fluctuate around $\beta$ with ever smaller variance for growing $n$).

Improve this answer
$\endgroup$
4
  • $\begingroup$ then the conclusion is that $\beta$ of the CIs will contain it's respective true parameter $\theta_i$. I am trying to see how this removes the need to introduce the idea of repeating the same experiment over and over. Is it correct to say , for example, true parameter of experiment 1 is in it's CI because this experiment can be one of the 90% experiments who CI contain it's true parameter but how can we compute that probability that experiment 1 is in that 90%? is it that we assume the success for each experiment is equally likely and it's $beta$ @Christian Hennig $\endgroup$
    – Mina
    Commented Feb 21, 2022 at 21:50
  • $\begingroup$ I'm not quite sure I understand your problem. Indeed we assume the success for each experiment is equally likely and it's $\beta$ (definition of the level of a CI). I wonder whether your problem is more a philosophical one. You could argue that assuming $P\{\theta_i\in C_i\}=\beta$ as a frequentist probability requires replication of the same experiment. That's a fair point. For me the frequentist definition of probability should be interpreted as "we think of an experiment as if..."-idealisation, rather than this really happening. $\endgroup$
    – Christian Hennig
    Commented Feb 21, 2022 at 23:07
  • $\begingroup$ perfect! and yes, I indeed was struggling with the fact that 𝑃{𝜃𝑖∈𝐶𝑖}=𝛽 as a frequentist probability requires replication of the same experiment. Thanks for your explanation. $\endgroup$
    – Mina
    Commented Feb 22, 2022 at 2:13
  • $\begingroup$ @Mina This is a general issue with frequentist probability, not with confidence intervals in particular. $\endgroup$
    – Christian Hennig
    Commented Feb 23, 2022 at 10:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.