Questions tagged [coverage-probability]
The probability that a confidence interval actually contains the true parameter.
53
questions
0
votes
0
answers
95
views
Is the empirical coverage rate of a confidence interval equal to the test's acceptance rate for a correct estimate (in a simulation study)?
I am conducting a simulation study to assess the performance of several confidence intervals.
My approach is to simulated (N = 10000) datasets, estimate the parameters, construct a confidence interval,...
- 23
1
vote
0
answers
48
views
How to transform percentage cover data into species abundance data in algae
I have collected percentage data for Rocky shore and summarised it per algae type like the example below:
...
- 11
0
votes
0
answers
22
views
misspecified models coverage and efficiency
Suppose I have model $M$ generating data $Y=\beta_0+\beta_1X+\beta_2Z+\beta_3W$ with all $\beta$'s known. Instead of using model $M$, I used misspecified models $M':Y=\beta'_0+\beta'_1X+\beta'_2Z$, $M'...
- 1,365
5
votes
0
answers
168
views
Why beta regression?
According to http://r-statistics.co/Beta-Regression-With-R.html, the topline remark is:
Beta regression is used when you want to model Y that are probabilities themselves
Grammar aside, one may ...
- 60.7k
2
votes
0
answers
36
views
How to estimate sampling coverage of a sample from unknown categorical distribution?
Sorry it seems a pretty basic question, but I cannot find any clue in my probability theory and statistics textbooks.
Let $$X\in D$$ be a categorical random variable of unknown distribution P on ...
- 21
1
vote
0
answers
101
views
Scoring Probabilistic Forecasts - Can we infer a standard deviation from the 84.1 quantile prediction?
I am trying to compare forecasts of a series, and have several trained estimators which are deep neural networks with arbitrary architecture.
I'd like to compare the accuracy of their probabilistic ...
1
vote
2
answers
207
views
Using remaining useful life predictions to assess model fit with right censored data
I am modelling time to failure of some units. All units are made my the same manufacturer and there are no recorded covariates to distinguish between the units.
This means that the central, $95\%$ say,...
- 843
11
votes
2
answers
3k
views
Random forests confidence intervals and prediction
This is a short simulation to check the coverage, when used as predictive intervals, of the random forest confidence intervals introduced in the paper:
S. Wager, T. Hastie and B. Efron. Confidence ...
- 23.2k
3
votes
0
answers
76
views
How to interpret a n-related change in coverage for model (simulation study)
I have repeated measures data from n_subjects where each has n_obs number of measurements before and after some intervention, ...
- 133
5
votes
3
answers
2k
views
Coverage for confidence intervals
I would like to know if my simulation approach to find the coverage for a confidence interval of a prediction $\boldsymbol{\beta}^T\boldsymbol{X}_N$ is correct
I generated a dataset of $n$ samples of ...
- 177
13
votes
2
answers
1k
views
Not getting 95% coverage for 95% t-distribution CI
I'm simulating a bunch of 95% confidence intervals on samples taken from a normal distribution. Since the data is normal, then, I think, my 95% confidence should translate into a 95% coverage ...
- 2,217
1
vote
0
answers
60
views
What are the implications of a low coverage in multiple imputation?
When testing multiple imputation algorithms in simulations, the bias of the examined estimates and the 95% coverage rate are often used as a quality metric. I understand that it is generally ...
- 11
1
vote
1
answer
175
views
Confidence Interval of Coverage Probabilities
Does it make much sense to calculate confidence intervals for coverage probabilities?
For instance, if a CI coverage probability is estimated to be 0.80, does computing an exact 95% binomial ...
- 1,537
0
votes
1
answer
305
views
Coverage of a Bootstrap Confidence Interval for a Change in a Binomial Proportion
How can I estimate the coverage of a bootstrap confidence interval for a change in a binomial proportion please?
For example, if the results from two tests (A and B) are:
...
- 179
1
vote
0
answers
136
views
Coverage of the C.I for the variance on a non-normal population
Consider the confidence interval for the population variance $\sigma^2$ constructed under normality: $$\left[A_n,B_n\right]=\left[\frac{(n-1)s^2}{\chi^2_{n-1,\alpha/2}},\frac{(n-1)s^2}{\chi^2_{n-1,1-\...
- 111
1
vote
0
answers
288
views
Poor Coverage for Bootstrap Confidence Interval on Mean
Based on a highly skewed population, I was hoping to obtain good coverage with bootstrap (vs standard) confidence intervals. I’m not having much luck.
I specify my population in R as a Gamma ...
- 53
5
votes
1
answer
139
views
Why is the coverage of this lme4 confidence interval less than 95%?
I have data that can be described using the model $y_{i j} = \alpha_i + \epsilon_{i j}$, where $\alpha_i \sim \text{N}(\mu_{\alpha}, \sigma_{\alpha}^2)$ and $\epsilon_{i j} \sim \text{N}(0, \sigma_{\...
- 83
0
votes
1
answer
220
views
Interval Estimation for a Binomial Proportion Given a Specific Test Outcome
Imagine that I had a coin, I tossed it 10 times (n) and it came up heads each time (x). What proportion heads I would get if I tossed it infinity times? A point estimate is 100%.
I can get the ...
- 179
2
votes
0
answers
879
views
How to calculate coverage probability from binomial confidence intervals?
For a given alpha I have been calculating various confidence intervals for a binomial distribution(Wald, Wilson, Agrest-Coulla etc.):
...
- 123
1
vote
1
answer
126
views
VaR backtesting: counting the number of rejections
Let's say you calculate the number of VaR rejections for every $r_{1,t}$, $r_{2,t}$,
should you have the same number of rejections in a model, irrespectively of the weights being different?
$[w,\ 1-w]...
1
vote
1
answer
97
views
How can I compare the lengths between these two confidence intervals? [closed]
Let $X\sim Beta(\theta,1)$. I was asked to find two confidence intervals and compare their lengths.
The first confidence interval is for $Y=-(logX)^{-1}$ over the set $[y/2,y]$.
The second ...
- 1,829
0
votes
0
answers
62
views
Test if confidence interval by observation is correct
I have calculated a "confidence interval" of the days between purchase for 600K users. The exact method I used isn't relevant but a simple version would be for each user calculate the distribution ...
0
votes
1
answer
25
views
internal process of proc mianalyze procedure [closed]
Does proc mi analyze in sas consider only estimates of imputed values or complete data sets that is imputed values as well as non imputed values?
- 13
1
vote
0
answers
131
views
Coverage probability and the nominal coverage probability frequently
What is the difference between coverage probability and the nominal coverage probability frequently?
- 81
1
vote
1
answer
1k
views
Coverage probability for Wald confidence interval with small sample size
a) My understanding is that the sample variance (i.e. the squared deviation divided by n - 1 instead of n) constitutes a mean-unbiased estimator of population variance. However, I've run multiple ...
- 1,404
10
votes
1
answer
5k
views
Posterior variance vs variance of the posterior mean
This question is about the frequentist properties of Bayesian methods.
Suppose we have data ${\bf y}$ generated from a distribution with a single parameter $\theta$, equipped with a prior $\pi(\theta)...
- 7,077
0
votes
0
answers
137
views
Conditional coverage probability
Suppose I'm interested in a population mean $\mu$ of some finite discrete variable $Y \sim P$ defined on $\mathbb{R}$. I have a $100(1-\alpha)\%$ confidence interval for the sample estimate $\hat{\mu}$...
- 11
4
votes
2
answers
111
views
coverage index?
Suppose I have a space of potential outcomes X with a probability distribution on it. I assume that there is a distance function between elements of X (e.g. X is a metric space). I also have a set S ...
- 549
0
votes
0
answers
609
views
Coverage probability for the Bayesian credible interval for Normal distribution
Bayesian Inference for the Normal Distribution, I use the following r code to obtain the posterior distribution. Let's say the data, $X\sim N(\mu, \sigma^{2})$ and $\mu \sim N(0,10)$ and $\sigma \sim \...
- 335
6
votes
3
answers
530
views
Joint credible regions from MCMC draws
Lets say I have $n$ posterior samples of $\theta_1$ and $\theta_2$. I suppose that any region $R$ which contains exactly $(1-\alpha)n$ of the points will be an approximate $(1-\alpha)\times100$ ...
- 7,077
3
votes
2
answers
542
views
Econometrics: What will happen if I have a biased estimator (either positively or negatively biased) when constructing the confidence interval
Suppose that the OLS estimator for $\beta_j$ is in fact biased. How would this affect the use of confidence intervals as hypothesis testing tools (assuming a two-tailed test)? That is, what sort of ...
- 29
1
vote
1
answer
89
views
Interpretation of values contained in a confidence interval [duplicate]
Are the values within a confidence interval those that do not significantly differ from the point estimate? Or, put differently, how do we interpret the values contained in a CI given that the CI is ...
- 393
4
votes
1
answer
162
views
convergence in probability for symmetric beta density
I am trying to solve the following problem.
Prove that for $X_n\sim \operatorname{Beta}(n,n)$ , $X_n $ converges in probability to $\frac {1}{2}$.
This is what I tried:
Since as $ n\rightarrow \...
- 643
1
vote
0
answers
186
views
Comparing the results of four missing data imputation methods
Below are the Bias, MSE and Coverage of four missing data imputation methods at different percentages of missing values. From these we can see that, m4 method gives ...
- 83
2
votes
2
answers
180
views
Assessing model prediction performance using tolerance interval coverage
I am trying to assess a model's prediction performance, and one metric I look at is the percentage of new observations that fall within the 95% tolerance interval, and see whether or not it is ...
- 43
0
votes
1
answer
186
views
Number of replications needed for coverage probability study
I am constructing confidence interval based on Wald, likelihood ratio and jackknife for the parameters of a model via simulation. To evaluate which methods is preferable for the parameters, I did a ...
- 1
2
votes
1
answer
3k
views
Second order inclusion probabilities in With-Replacement Sampling
I'm reading the book "Model Assisted Survey Sampling" from Särndal et al.
In chapter 2, there's a section about Sampling with replacement. I'll put this into context:
We have $m$ independent draws, ...
- 373
1
vote
1
answer
320
views
Underestimated Coverage probability
Let $U0$ denotes intercept variance and $U1$ denotes slope variance.
Given that the coverage rate for the intercept variance is $91$% $(U0)$ , and the coverage rate for the slope variance is $91.2$% $...
- 771
21
votes
2
answers
1k
views
confidence intervals' coverage with regularized estimates
Suppose I'm trying to estimate a large number of parameters from some high-dimensional data, using some kind of regularized estimates. The regularizer introduces some bias into the estimates, but it ...
- 11.6k
0
votes
2
answers
2k
views
coverage of confidence interval
I need to check the coverage of a confidence interval, but I don't know which one of the following approaches I should use.
Approach 1:
Estimate the regression parameter $\theta$
Use the sandwich ...
- 16
2
votes
0
answers
260
views
Confidence Intervals of a Given Width
I'm working through some questions on confidence intervals. My answer doesn't match the book, but the book's answer was a number I had a few steps before the end.
I have ten numbers which are a ...
- 141
12
votes
3
answers
842
views
Regarding convergence in probability
Let $\{X_n\}_{n\geq 1}$ be a sequence of random variables s.t $X_n \to a$ in probability, where $a>0$ is a fixed constant. I'm trying to show the following:
$$\sqrt{X_n} \to \sqrt{a}$$
and
$$\frac{...
- 361
0
votes
1
answer
365
views
Convergence of Sum of Sums of random variables : trivial?
I have a sequence of i.i.d random varaibles $X_1, X_2, ...$ with finite mean $\mu$ and finite variance.
I also have another sequence of i.i.d random varaibles $Y_1, Y_2, ...$ with the same finite mean ...
- 3
2
votes
0
answers
182
views
Coverage probability from fiducial/confidence distribution
I am trying to realize how to compute confidence intervals from a fiducial distribution/confidence distribution with possibly more than one parameter. But for now I would just like to understand if I ...
- 123
1
vote
0
answers
85
views
Probability distribution of three random variables
The following formula is a formula I got from a paper that deals with wireless networks specifically when calculating coverage probabilities, it is powerful because one can use it for general ...
- 205
2
votes
1
answer
815
views
Calculating the mean excess loss
Suppose $X$ has the following pdf:
$$ f_x(x)=0.01 \qquad for\space 0\le x<100$$
Find the pdf of $X_p$ (the excess-loss variable) and calculate the mean excess loss for $d=10$.
\begin{align}
P(X_p&...
- 463
1
vote
0
answers
92
views
error on truncated rms
I am computing the RMS of a sample to estimate the standar error $\sigma$ of the underlying distribution (for simplicity let say a normal distribution $N[\mu$, $\sigma$]).
$ \text{RMS} = \sum_{i=1}^N ...
- 726
7
votes
1
answer
852
views
Why (mathematically) is the parametric bootstrap usually better than the empirical one?
As I know from experience, the parametric bootstrap performs better in terms of coverage probability for confidence intervals then the empirical bootstrap.
Of course, this makes sense because you put ...
- 484
0
votes
1
answer
396
views
Inconsistency between RMSE and 95% CI Coverage
I am running simulations to compare different weighting methods to estimate the mean of y (with missing values). I use bias, RMSE and 95% CI Coverage as my performance metrics. However, looking at ...
- 158
1
vote
1
answer
561
views
Confidence Interval Coverage-error and Type I error
Could somebody explain to me the relationship between coverage error and type one errors in multiple comparisons testing, if there is one in fact? Does a coverage error occur when the true value of ...
- 51