Questions tagged [confidence-interval]

A confidence interval is an interval that covers an unknown parameter with $(1-\alpha)\%$ confidence. Confidence intervals are a frequentist concept. They are often confused with credible intervals which is the Bayesian analog.

Filter by
Sorted by
Tagged with
283 votes
16 answers
96k views

Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
user avatar
54 votes
4 answers
17k views

Why do statisticians say a non-significant result means "you can't reject the null" as opposed to accepting the null hypothesis?

Traditional statistical tests, like the two sample t-test, focus on trying to eliminate the hypothesis that there is no difference between a function of two independent samples. Then, we choose a ...
user avatar
  • 2,250
40 votes
3 answers
14k views

Relation between confidence interval and testing statistical hypothesis for t-test

It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's ...
user avatar
  • 1,251
296 votes
10 answers
170k views

What's the difference between a confidence interval and a credible interval?

Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
user avatar
  • 5,637
59 votes
4 answers
34k views

Confidence interval for Bernoulli sampling

I have a random sample of Bernoulli random variables $X_1 ... X_N$, where $X_i$ are i.i.d. r.v. and $P(X_i = 1) = p$, and $p$ is an unknown parameter. Obviously, one can find an estimate for $p$: $\...
user avatar
103 votes
12 answers
17k views

What, precisely, is a confidence interval?

I know roughly and informally what a confidence interval is. However, I can't seem to wrap my head around one rather important detail: According to Wikipedia: A confidence interval does not ...
user avatar
  • 7,445
120 votes
6 answers
185k views

Difference between confidence intervals and prediction intervals

For a prediction interval in linear regression you still use $\hat{E}[Y|x] = \hat{\beta_0}+\hat{\beta}_{1}x$ to generate the interval. You also use this to generate a confidence interval of $E[Y|x_0]$....
user avatar
  • 1,367
47 votes
2 answers
16k views

Confidence interval around binomial estimate of 0 or 1

What is the best technique to calculate a confidence interval of a binomial experiment, if your estimate is that $p=0$ (or similarly $p=1$) and sample size is relatively small, for example $n=25$?
user avatar
  • 3,079
89 votes
3 answers
42k views

Shape of confidence interval for predicted values in linear regression

I have noticed that the confidence interval for predicted values in an linear regression tends to be narrow around the mean of the predictor and fat around the minimum and maximum values of the ...
user avatar
  • 12.4k
94 votes
9 answers
13k views

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic: Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
user avatar
15 votes
2 answers
5k views

Biased bootstrap: is it okay to center the CI around the observed statistic?

This is similar to Bootstrap: estimate is outside of confidence interval I have some data that represents counts of genotypes in a population. I want to estimate genetic diversity using Shannon's ...
user avatar
  • 201
105 votes
3 answers
18k views

What are examples where a "naive bootstrap" fails?

Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of ...
user avatar
  • 9,200
49 votes
14 answers
6k views

Clarification on interpreting confidence intervals?

My current understanding of the notion "confidence interval with confidence level $1 - \alpha$" is that if we tried to calculate the confidence interval many times (each time with a fresh sample), it ...
user avatar
  • 543
32 votes
3 answers
26k views

What is a Highest Density Region (HDR)?

In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book. One similar term is the Highest Posterior ...
user avatar
17 votes
2 answers
17k views

Confidence intervals for median

I have a distribution of samples with a small number of values in each one (less than $10$). I have calculated the median for each sample, which I want to compare with a model and obtain the ...
user avatar
  • 423
59 votes
5 answers
21k views

Is it true that the percentile bootstrap should never be used?

In the MIT OpenCourseWare notes for 18.05 Introduction to Probability and Statistics, Spring 2014 (currently available here), it states: The bootstrap percentile method is appealing due to its ...
user avatar
  • 3,883
5 votes
2 answers
3k views

Why is there a need for a 'sampling distribution' to find confidence intervals?

I understand the key principles behind confidence intervals, but there's something I want a bit of clarification on. Let's say I have a basket of apples that I picked at the orchard. The weight is ...
user avatar
  • 644
10 votes
2 answers
2k views

Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
user avatar
  • 183
30 votes
2 answers
2k views

Is rejecting the hypothesis using p-value equivalent to hypothesis not belonging to the confidence interval?

While formally deriving the confidence interval of an estimate, I ended up with a formula that resembles very closely the way $p$-value is computed. Thus the question: are they formally equivalent? I....
user avatar
  • 1,219
12 votes
1 answer
2k views

Estimating parameters for a binomial

First of all I'd like to precise that I'm not an expert of the subject. Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
user avatar
15 votes
3 answers
58k views

How to choose a confidence level?

I often use a 90% confidence level, accepting that this has a greater degree of uncertainty than 95% or 99%. But are there any guidelines on how to choose the right confidence level? Or guidelines for ...
user avatar
30 votes
3 answers
76k views

What is the difference between confidence intervals and hypothesis testing?

I have read about controversies regarding hypothesis testing with some commentators suggesting that hypothesis testing should not be used. Some commentators suggest that confidence intervals should ...
user avatar
  • 1,129
22 votes
1 answer
40k views

Why is chi square used when creating a confidence interval for the variance?

This is a very basic question. Why do we use a chi square distribution? What is the meaning of this distribution? Why is this the distribution used for creating a confidence interval for the variance? ...
user avatar
  • 665
63 votes
3 answers
26k views

Explain the xkcd jelly bean comic: What makes it funny?

I see that one time out of the twenty total tests they run, $p < 0.05$, so they wrongly assume that during one of the twenty tests, the result is significant ($0.05 = 1/20$). xkcd jelly bean ...
user avatar
  • 693
46 votes
4 answers
130k views

How to calculate a confidence level for a Poisson distribution?

Would like to know how confident I can be in my $\lambda$. Anyone know of a way to set upper and lower confidence levels for a Poisson distribution? Observations ($n$) = 88 Sample mean ($\lambda$) =...
user avatar
  • 741
24 votes
2 answers
30k views

How to obtain a confidence interval for a percentile?

I have a bunch of raw data values that are dollar amounts and I want to find a confidence interval for a percentile of that data. Is there a formula for such a confidence interval?
user avatar
  • 802
15 votes
1 answer
3k views

Prediction interval for binomial random variable

What is the formula (approximate or exact) for a prediction interval for a Binomial random variable? Assume $Y \sim \mathsf{Binom}(n, p)$, and we observe $y$ (drawn from $Y$). The $n$ is known. Our ...
user avatar
38 votes
6 answers
6k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
user avatar
  • 723
20 votes
1 answer
11k views

Should confidence intervals for linear regression coefficients be based on the normal or $t$ distribution?

Let's have some linear model, for example just simple ANOVA: ...
user avatar
  • 5,835
11 votes
1 answer
2k views

Does invertible monotone transformation of a confidence interval give you a confidence interval (at the same level) in the transformed space?

Suppose $$ (a,b) $$ is a $(1-\alpha)$ level confidence interval for a parameter $\theta$. Suppose $\eta$ is a monotone invertible transformation. Then, is $$ \left (\eta(a), \eta(b) \right ) $$ ...
user avatar
  • 191
2 votes
2 answers
2k views

Why and how does adding an interaction term affects the confidence interval of a main effect?

I was wondering why, in a regression model including an interaction term, the confidence interval of one of the main effects becomes wider. Consider this Cox regression, where the variable IR_BMI27 ...
user avatar
  • 275
59 votes
4 answers
8k views

Are all values within a 95% confidence interval equally likely?

I have found discordant information on the question: "If one constructs a 95% confidence interval (CI) of a difference in means or a difference in proportions, are all values within the CI equally ...
user avatar
  • 5,573
48 votes
3 answers
34k views

Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?

Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
user avatar
  • 5,538
29 votes
4 answers
30k views

How to compute the confidence interval of the ratio of two normal means

I want to derive the limits for the $100(1-\alpha)\%$ confidence interval for the ratio of two means. Suppose, $X_1 \sim N(\theta_1, \sigma^2)$ and $X_2 \sim N(\theta_2, \sigma^2)$ being independent, ...
user avatar
31 votes
6 answers
16k views

Binomial confidence interval estimation - why is it not symmetric?

I have used the following r code to estimate the confidence intervals of a binomial proportion because I understand that that substitutes for a "power calculation" when designing receiver operating ...
user avatar
  • 5,505
10 votes
1 answer
587 views

The basic logic of constructing a confidence interval

Consider a model with a parameter of interest, $\theta$, and its point estimator, $\hat\theta$. For simplicity, assume $\hat\theta\sim N(\theta,\sigma^2/n)$ (in numerous instances this could be ...
user avatar
27 votes
4 answers
27k views

Confidence interval of RMSE

I have taken a sample of $n$ data points from a population. Each of these points has a true value (known from ground truth) and an estimated value. I then calculate the error for each sampled point ...
user avatar
  • 2,007
11 votes
2 answers
3k views

Does it make sense to compute confidence intervals and to test hypotheses when data from whole population is available?

Does it make sense to compute confidence intervals and to test hypotheses when data from the whole population is available? In my opinion, the answer is no, since we can accurately compute the true ...
user avatar
  • 3,669
21 votes
1 answer
14k views

Computing prediction intervals for logistic regression

I would like to understand how to generate prediction intervals for logistic regression estimates. I was advised to follow the procedures in Collett's Modelling Binary Data, 2nd Ed p.98-99. After ...
user avatar
55 votes
6 answers
72k views

How to determine best cutoff point and its confidence interval using ROC curve in R?

I have the data of a test that could be used to distinguish normal and tumor cells. According to ROC curve it looks good for this purpose (area under curve is 0.9): My questions are: How to ...
user avatar
27 votes
1 answer
14k views

Constructing confidence intervals based on profile likelihood

In my elementary statistics course, I learnt how to construct 95% confidence interval such as population mean, $\mu$, based on asymptotic normality for "large" sample sizes. Apart from resampling ...
user avatar
13 votes
2 answers
2k views

Can we use bootstrap samples that are smaller than original sample?

I want to use bootstrapping to estimate confidence intervals for estimated parameters from a panel dataset with N=250 firms and T=50 month. The estimation of parameters is computationally expensive (...
user avatar
  • 161
11 votes
3 answers
7k views

What does a confidence interval (vs. a credible interval) actually express? [duplicate]

Possible Duplicate: What, precisely, is a confidence interval? Yes, similar questions have been asked before, but many of the answers seem contradictory and don't address my issue. (Or my ...
user avatar
  • 211
40 votes
2 answers
36k views

Understanding shape and calculation of confidence bands in linear regression

I am trying to understand the origin of the curved shaped of confidence bands associated with an OLS linear regression and how it relates to the confidence intervals of the regression parameters (...
user avatar
  • 401
42 votes
5 answers
34k views

Confidence interval for median

I have to find a 95% C.I. on the median and other percentiles. I don't know how to approach this. I mainly use R as a programming tool.
user avatar
36 votes
4 answers
14k views

Are smaller p-values more convincing?

I've been reading up on $p$-values, type 1 error rates, significance levels, power calculations, effect sizes and the Fisher vs Neyman-Pearson debate. This has left me feeling a bit overwhelmed. I ...
user avatar
  • 1,596
34 votes
3 answers
5k views

How to find confidence intervals for ratings?

Evan Miller's "How Not to Sort by Average Rating" proposes using the lower bound of a confidence interval to get a sensible aggregate "score" for rated items. However, it's working with a Bernoulli ...
user avatar
36 votes
2 answers
4k views

Is there a reliable nonparametric confidence interval for the mean of a skewed distribution?

Very skewed distributions such as the log-normal do not result in accurate bootstrap confidence intervals. Here is an example showing that the left and right tail areas are far from the ideal 0.025 ...
user avatar
19 votes
1 answer
10k views

Bootstrap-based confidence interval

While studying bootstrap-based confidence interval, I once read the following statement: If the bootstrap distribution is skewed to the right, the bootstrap-based confidence interval incorporates a ...
user avatar
  • 4,672
10 votes
1 answer
3k views

Discrete functions: Confidence interval coverage?

How to calculate discrete interval coverage? What I know how to do: If I had a continuous model, I could define a 95% confidence interval for each of my predicted values, and then see how often the ...
user avatar
  • 3,363

1
2 3 4 5
12