Questions tagged [coverage-probability]

The probability that a confidence interval actually contains the true parameter.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Conservative coverage probability when an biased estimator is used for the variance

Suppose that $X_n\sim N(\mu, \sigma^2_n)$. Thus, to construct a 95% CI for $\mu$, we can use $X_n\pm 1.96 \sigma_n$. The coverage probability, $P(\mu\in [X_n\pm 1.96 \sigma_n])$, is equal to 95%. ...
user avatar
  • 789
9 votes
2 answers
439 views

Random forests confidence intervals and prediction

This is a short simulation to check the coverage, when used as predictive intervals, of the random forest confidence intervals introduced in the paper: S. Wager, T. Hastie and B. Efron. Confidence ...
user avatar
  • 22k
0 votes
0 answers
37 views

Small bias but low coverage probability

I tried my codes on 100 and 500 simulation data. Each data has 100, 500, or 1000 observations. The number of simulated data did not affect much of my results. However as I increased the number of ...
user avatar
  • 41
0 votes
0 answers
5 views

Why does an operational domain isn't an MV-set and why is it a problem for images and computer vision?

I am working on formalizing the operational domain of an algorithm. If the operational domain is the set $D=\{x, g(x)=1\}$ where $g$ is the selector that says when the predictor $f$ can be applied. $g(...
user avatar
0 votes
0 answers
7 views

Why does the statistical nature of the coverage of an operational domain mathematical formalisation is a problem for images and computer vision?

I am working on formalizing the operational domain of an algorithm. If the operational domain is the set $D=\{x, g(x)=1\}$ where $g$ is the selector that says when the predictor $f$ can be applied. $g(...
user avatar
3 votes
0 answers
72 views

How to interpret a n-related change in coverage for model (simulation study)

I have repeated measures data from n_subjects where each has n_obs number of measurements before and after some intervention, ...
user avatar
3 votes
3 answers
619 views

Coverage for confidence intervals

I would like to know if my simulation approach to find the coverage for a confidence interval of a prediction $\boldsymbol{\beta}^T\boldsymbol{X}_N$ is correct I generated a dataset of $n$ samples of ...
user avatar
  • 157
13 votes
2 answers
992 views

Not getting 95% coverage for 95% t-distribution CI

I'm simulating a bunch of 95% confidence intervals on samples taken from a normal distribution. Since the data is normal, then, I think, my 95% confidence should translate into a 95% coverage ...
user avatar
  • 2,047
1 vote
0 answers
17 views

What are the implications of a low coverage in multiple imputation?

When testing multiple imputation algorithms in simulations, the bias of the examined estimates and the 95% coverage rate are often used as a quality metric. I understand that it is generally ...
user avatar
0 votes
0 answers
50 views

Bivariate normal PDF and it's confidence interval coverage

I'im currently studying the confidence interval where the PDF of (X, Y) is a bivariate normal PDF. After calculating the confidence interval, i am calculating the coverage too. Now my question is: ...
user avatar
  • 1
0 votes
1 answer
68 views

Confidence Interval of Coverage Probabilities

Does it make much sense to calculate confidence intervals for coverage probabilities? For instance, if a CI coverage probability is estimated to be 0.80, does computing an exact 95% binomial ...
user avatar
  • 1,057
0 votes
1 answer
164 views

Coverage of a Bootstrap Confidence Interval for a Change in a Binomial Proportion

How can I estimate the coverage of a bootstrap confidence interval for a change in a binomial proportion please? For example, if the results from two tests (A and B) are: ...
user avatar
  • 179
1 vote
0 answers
84 views

Coverage of the C.I for the variance on a non-normal population

Consider the confidence interval for the population variance $\sigma^2$ constructed under normality: $$\left[A_n,B_n\right]=\left[\frac{(n-1)s^2}{\chi^2_{n-1,\alpha/2}},\frac{(n-1)s^2}{\chi^2_{n-1,1-\...
user avatar
  • 111
1 vote
0 answers
157 views

Poor Coverage for Bootstrap Confidence Interval on Mean

Based on a highly skewed population, I was hoping to obtain good coverage with bootstrap (vs standard) confidence intervals. I’m not having much luck. I specify my population in R as a Gamma ...
user avatar
  • 43
6 votes
1 answer
129 views

Why is the coverage of this lme4 confidence interval less than 95%?

I have data that can be described using the model $y_{i j} = \alpha_i + \epsilon_{i j}$, where $\alpha_i \sim \text{N}(\mu_{\alpha}, \sigma_{\alpha}^2)$ and $\epsilon_{i j} \sim \text{N}(0, \sigma_{\...
user avatar
0 votes
1 answer
163 views

Interval Estimation for a Binomial Proportion Given a Specific Test Outcome

Imagine that I had a coin, I tossed it 10 times (n) and it came up heads each time (x). What proportion heads I would get if I tossed it infinity times? A point estimate is 100%. I can get the ...
user avatar
  • 179
1 vote
0 answers
544 views

How to calculate coverage probability from binomial confidence intervals?

For a given alpha I have been calculating various confidence intervals for a binomial distribution(Wald, Wilson, Agrest-Coulla etc.): ...
user avatar
  • 113
1 vote
1 answer
97 views

VaR backtesting: counting the number of rejections

Let's say you calculate the number of VaR rejections for every $r_{1,t}$, $r_{2,t}$, should you have the same number of rejections in a model, irrespectively of the weights being different? $[w,\ 1-w]...
user avatar
1 vote
1 answer
53 views

How can I compare the lengths between these two confidence intervals? [closed]

Let $X\sim Beta(\theta,1)$. I was asked to find two confidence intervals and compare their lengths. The first confidence interval is for $Y=-(logX)^{-1}$ over the set $[y/2,y]$. The second ...
user avatar
  • 1,595
0 votes
0 answers
57 views

Test if confidence interval by observation is correct

I have calculated a "confidence interval" of the days between purchase for 600K users. The exact method I used isn't relevant but a simple version would be for each user calculate the distribution ...
user avatar
0 votes
1 answer
18 views

internal process of proc mianalyze procedure [closed]

Does proc mi analyze in sas consider only estimates of imputed values or complete data sets that is imputed values as well as non imputed values?
user avatar
1 vote
0 answers
85 views

Coverage probability and the nominal coverage probability frequently

What is the difference between coverage probability and the nominal coverage probability frequently?
user avatar
1 vote
1 answer
696 views

Coverage probability for Wald confidence interval with small sample size

a) My understanding is that the sample variance (i.e. the squared deviation divided by n - 1 instead of n) constitutes a mean-unbiased estimator of population variance. However, I've run multiple ...
user avatar
  • 954
10 votes
1 answer
3k views

Posterior variance vs variance of the posterior mean

This question is about the frequentist properties of Bayesian methods. Suppose we have data ${\bf y}$ generated from a distribution with a single parameter $\theta$, equipped with a prior $\pi(\theta)...
user avatar
  • 6,118
0 votes
0 answers
80 views

Conditional coverage probability

Suppose I'm interested in a population mean $\mu$ of some finite discrete variable $Y \sim P$ defined on $\mathbb{R}$. I have a $100(1-\alpha)\%$ confidence interval for the sample estimate $\hat{\mu}$...
user avatar
4 votes
2 answers
89 views

coverage index?

Suppose I have a space of potential outcomes X with a probability distribution on it. I assume that there is a distance function between elements of X (e.g. X is a metric space). I also have a set S ...
user avatar
  • 551
0 votes
0 answers
455 views

Coverage probability for the Bayesian credible interval for Normal distribution

Bayesian Inference for the Normal Distribution, I use the following r code to obtain the posterior distribution. Let's say the data, $X\sim N(\mu, \sigma^{2})$ and $\mu \sim N(0,10)$ and $\sigma \sim \...
user avatar
  • 335
5 votes
3 answers
374 views

Joint credible regions from MCMC draws

Lets say I have $n$ posterior samples of $\theta_1$ and $\theta_2$. I suppose that any region $R$ which contains exactly $(1-\alpha)n$ of the points will be an approximate $(1-\alpha)\times100$ ...
user avatar
  • 6,118
3 votes
2 answers
394 views

Econometrics: What will happen if I have a biased estimator (either positively or negatively biased) when constructing the confidence interval

Suppose that the OLS estimator for $\beta_j$ is in fact biased. How would this affect the use of confidence intervals as hypothesis testing tools (assuming a two-tailed test)? That is, what sort of ...
user avatar
1 vote
1 answer
79 views

Interpretation of values contained in a confidence interval [duplicate]

Are the values within a confidence interval those that do not significantly differ from the point estimate? Or, put differently, how do we interpret the values contained in a CI given that the CI is ...
user avatar
4 votes
1 answer
133 views

convergence in probability for symmetric beta density

I am trying to solve the following problem. Prove that for $X_n\sim \operatorname{Beta}(n,n)$ , $X_n $ converges in probability to $\frac {1}{2}$. This is what I tried: Since as $ n\rightarrow \...
user avatar
  • 613
1 vote
0 answers
174 views

Comparing the results of four missing data imputation methods

Below are the Bias, MSE and Coverage of four missing data imputation methods at different percentages of missing values. From these we can see that, m4 method gives ...
user avatar
  • 63
2 votes
2 answers
161 views

Assessing model prediction performance using tolerance interval coverage

I am trying to assess a model's prediction performance, and one metric I look at is the percentage of new observations that fall within the 95% tolerance interval, and see whether or not it is ...
user avatar
0 votes
1 answer
162 views

Number of replications needed for coverage probability study

I am constructing confidence interval based on Wald, likelihood ratio and jackknife for the parameters of a model via simulation. To evaluate which methods is preferable for the parameters, I did a ...
user avatar
  • 1
1 vote
1 answer
2k views

Second order inclusion probabilities in With-Replacement Sampling

I'm reading the book "Model Assisted Survey Sampling" from Särndal et al. In chapter 2, there's a section about Sampling with replacement. I'll put this into context: We have $m$ independent draws, ...
user avatar
  • 283
1 vote
1 answer
274 views

Underestimated Coverage probability

Let $U0$ denotes intercept variance and $U1$ denotes slope variance. Given that the coverage rate for the intercept variance is $91$% $(U0)$ , and the coverage rate for the slope variance is $91.2$% $...
user avatar
  • 731
21 votes
2 answers
872 views

confidence intervals' coverage with regularized estimates

Suppose I'm trying to estimate a large number of parameters from some high-dimensional data, using some kind of regularized estimates. The regularizer introduces some bias into the estimates, but it ...
user avatar
0 votes
2 answers
1k views

coverage of confidence interval

I need to check the coverage of a confidence interval, but I don't know which one of the following approaches I should use. Approach 1: Estimate the regression parameter $\theta$ Use the sandwich ...
user avatar
  • 16
2 votes
0 answers
242 views

Confidence Intervals of a Given Width

I'm working through some questions on confidence intervals. My answer doesn't match the book, but the book's answer was a number I had a few steps before the end. I have ten numbers which are a ...
user avatar
12 votes
3 answers
508 views

Regarding convergence in probability

Let $\{X_n\}_{n\geq 1}$ be a sequence of random variables s.t $X_n \to a$ in probability, where $a>0$ is a fixed constant. I'm trying to show the following: $$\sqrt{X_n} \to \sqrt{a}$$ and $$\frac{...
user avatar
0 votes
1 answer
318 views

Convergence of Sum of Sums of random variables : trivial?

I have a sequence of i.i.d random varaibles $X_1, X_2, ...$ with finite mean $\mu$ and finite variance. I also have another sequence of i.i.d random varaibles $Y_1, Y_2, ...$ with the same finite mean ...
user avatar
  • 3
2 votes
0 answers
169 views

Coverage probability from fiducial/confidence distribution

I am trying to realize how to compute confidence intervals from a fiducial distribution/confidence distribution with possibly more than one parameter. But for now I would just like to understand if I ...
user avatar
  • 103
1 vote
0 answers
84 views

Probability distribution of three random variables

The following formula is a formula I got from a paper that deals with wireless networks specifically when calculating coverage probabilities, it is powerful because one can use it for general ...
user avatar
  • 205
2 votes
1 answer
698 views

Calculating the mean excess loss

Suppose $X$ has the following pdf: $$ f_x(x)=0.01 \qquad for\space 0\le x<100$$ Find the pdf of $X_p$ (the excess-loss variable) and calculate the mean excess loss for $d=10$. \begin{align} P(X_p&...
user avatar
  • 463
1 vote
0 answers
87 views

error on truncated rms

I am computing the RMS of a sample to estimate the standar error $\sigma$ of the underlying distribution (for simplicity let say a normal distribution $N[\mu$, $\sigma$]). $ \text{RMS} = \sum_{i=1}^N ...
user avatar
7 votes
1 answer
366 views

Why (mathematically) is the parametric bootstrap usually better than the empirical one?

As I know from experience, the parametric bootstrap performs better in terms of coverage probability for confidence intervals then the empirical bootstrap. Of course, this makes sense because you put ...
user avatar
0 votes
1 answer
351 views

Inconsistency between RMSE and 95% CI Coverage

I am running simulations to compare different weighting methods to estimate the mean of y (with missing values). I use bias, RMSE and 95% CI Coverage as my performance metrics. However, looking at ...
user avatar
  • 148
1 vote
1 answer
518 views

Confidence Interval Coverage-error and Type I error

Could somebody explain to me the relationship between coverage error and type one errors in multiple comparisons testing, if there is one in fact? Does a coverage error occur when the true value of ...
user avatar
2 votes
1 answer
419 views

Why are intervals for means narrower than intervals for individual observations?

I am looking at problem 5.01 relating to Galton's data on adult heights of fathers and sons, from Chapter 5 of Statistical Modelling: A Fresh Approach link to this book: http://www.mosaic-web.org/go/...
user avatar
  • 23
10 votes
3 answers
7k views

Is calculating "actual coverage probability" the same thing as calculating a "credible interval"?

I was reading an entry level statistics textbook. In the chapter on maximum likelihood estimation of the success proportion in data with binomial distribution, it gave a formula for calculating a ...
user avatar
  • 1,669