2
$\begingroup$

Suppose I observe $X_1, ..., X_n$ independent and identically distributed random variables and I calculate a confidence interval $(L, U)$ from this data. Now I take a second sample $Y_1, ..., Y_n$ from the same distribution. What is $P(L < \bar Y < U)$? How does it vary with $n$?

My initial thought was to use the delta method on $(\bar X, S^2)$, with

\begin{align} g \begin{pmatrix} \bar X \\ S^2 \end{pmatrix} = \begin{pmatrix} \displaystyle \bar X - t_{1 - \alpha /2, n - 1} {S \over \sqrt n} \\ \bar X \\ \displaystyle \bar X + t_{1 - \alpha /2, n - 1} {S \over \sqrt n} \end{pmatrix} \end{align}

But $\bar X$ and ${S \over \sqrt n}$ converge at different rates and I'm unsure how to handle this.

Finally, here's a quick simulation showing that for 10 samples from a standard normal, $P(L < \bar Y < U)$ is about 0.85:

n <- 10

coverage <- function() {
  x <- rnorm(n)
  ci <- t.test(x)$conf.int
  once <- function() dplyr::between(mean(rnorm(n)), ci[1], ci[2])
  mean(purrr::map_dbl(1:500, ~once()))
}

mean(purrr::map_dbl(1:500, ~coverage()))
#> [1] 0.846428

Created on 2019-03-16 by the reprex package (v0.2.1)

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ In general, this question might not even make any sense. For instance, when you construct a confidence interval for the variance of a distribution, you cannot validly compare values of the data to the variance--they are expressed in completely different units. You are attempting to use a confidence interval as if it were a prediction interval. This raises a question: what do you really need to accomplish? Do you need a procedure to construct a prediction interval or are you trying to learn something about interpreting confidence intervals (or neither)? $\endgroup$
    – whuber ♦
    Commented Mar 16, 2019 at 22:15
  • $\begingroup$ If you had a new sample, you could compute the new sample variance and compare it to an initial confidence interval for variance? I hadn't thought about prediction intervals because I was thinking about estimators rather than new data. Realizing a meta-analysis book is maybe the place to learn what a meaningful question is in this context. $\endgroup$
    – alexpghayes
    Commented Mar 16, 2019 at 22:55
  • 1
    $\begingroup$ There's no general answer to this. E.g. consider the absurd CI of 95% of the time the whole parameter space and 5% of the time the empty set. After seeing the CI you know whether the answer to your question is 100% or 0%. With other CIs this is not so clear, but similar effects may occur. $\endgroup$
    – Björn
    Commented Mar 17, 2019 at 6:59

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.