Questions tagged [confidence-interval]

A confidence interval is an interval that covers an unknown parameter with $(1-\alpha)\%$ confidence. Confidence intervals are a frequentist concept. They are often confused with credible intervals which is the Bayesian analog.

Filter by
Sorted by
Tagged with
266
votes
12answers
81k views

Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
48
votes
4answers
16k views

Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?

Traditional statistical tests, like the two sample t-test, focus on trying to eliminate the hypothesis that there is no difference between a function of two independent samples. Then, we choose a ...
36
votes
3answers
12k views

Relation between confidence interval and testing statistical hypothesis for t-test

It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's ...
276
votes
9answers
150k views

What's the difference between a confidence interval and a credible interval?

Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the difference between confidence intervals and credible intervals were the correct ones. How you would ...
93
votes
12answers
15k views

What, precisely, is a confidence interval?

I know roughly and informally what a confidence interval is. However, I can't seem to wrap my head around one rather important detail: According to Wikipedia: A confidence interval does not ...
53
votes
2answers
28k views

Confidence interval for Bernoulli sampling

I have a random sample of Bernoulli random variables $X_1 ... X_N$, where $X_i$ are i.i.d. r.v. and $P(X_i = 1) = p$, and $p$ is an unknown parameter. Obviously, one can find an estimate for $p$: $\...
104
votes
6answers
166k views

Difference between confidence intervals and prediction intervals

For a prediction interval in linear regression you still use $\hat{E}[Y|x] = \hat{\beta_0}+\hat{\beta}_{1}x$ to generate the interval. You also use this to generate a confidence interval of $E[Y|x_0]$....
44
votes
2answers
14k views

Confidence interval around binomial estimate of 0 or 1

What is the best technique to calculate a confidence interval of a binomial experiment, if your estimate is that $p=0$ (or similarly $p=1$) and sample size is relatively small, for example $n=25$?
14
votes
2answers
4k views

Biased bootstrap: is it okay to center the CI around the observed statistic?

This is similar to Bootstrap: estimate is outside of confidence interval I have some data that represents counts of genotypes in a population. I want to estimate genetic diversity using Shannon's ...
76
votes
2answers
36k views

Shape of confidence interval for predicted values in linear regression

I have noticed that the confidence interval for predicted values in an linear regression tends to be narrow around the mean of the predictor and fat around the minimum and maximum values of the ...
87
votes
7answers
11k views

Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals

A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic: Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
95
votes
3answers
16k views

What are examples where a “naive bootstrap” fails?

Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of ...
48
votes
14answers
6k views

Clarification on interpreting confidence intervals?

My current understanding of the notion "confidence interval with confidence level $1 - \alpha$" is that if we tried to calculate the confidence interval many times (each time with a fresh sample), it ...
5
votes
2answers
2k views

Why is there a need for a 'sampling distribution' to find confidence intervals?

I understand the key principles behind confidence intervals, but there's something I want a bit of clarification on. Let's say I have a basket of apples that I picked at the orchard. The weight is ...
28
votes
3answers
20k views

What is a Highest Density Region (HDR)?

In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book. One similar term is the Highest Posterior ...
52
votes
4answers
15k views

Is it true that the percentile bootstrap should never be used?

In the MIT OpenCourseWare notes for 18.05 Introduction to Probability and Statistics, Spring 2014 (currently available here), it states: The bootstrap percentile method is appealing due to its ...
30
votes
2answers
2k views

Is rejecting the hypothesis using p-value equivalent to hypothesis not belonging to the confidence interval?

While formally deriving the confidence interval of an estimate, I ended up with a formula that resembles very closely the way $p$-value is computed. Thus the question: are they formally equivalent? I....
12
votes
1answer
1k views

Estimating parameters for a binomial

First of all I'd like to precise that I'm not an expert of the subject. Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
14
votes
3answers
54k views

How to choose a confidence level?

I often use a 90% confidence level, accepting that this has a greater degree of uncertainty than 95% or 99%. But are there any guidelines on how to choose the right confidence level? Or guidelines for ...
29
votes
3answers
73k views

What is the difference between confidence intervals and hypothesis testing?

I have read about controversies regarding hypothesis testing with some commentators suggesting that hypothesis testing should not be used. Some commentators suggest that confidence intervals should ...
19
votes
1answer
38k views

Why is chi square used when creating a confidence interval for the variance?

This is a very basic question. Why do we use a chi square distribution? What is the meaning of this distribution? Why is this the distribution used for creating a confidence interval for the variance? ...
62
votes
3answers
24k views

Explain the xkcd jelly bean comic: What makes it funny?

I see that one time out of the twenty total tests they run, $p < 0.05$, so they wrongly assume that during one of the twenty tests, the result is significant ($0.05 = 1/20$). xkcd jelly bean ...
13
votes
2answers
11k views

Confidence intervals for median

I have a distribution of samples with a small number of values in each one (less than $10$). I have calculated the median for each sample, which I want to compare with a model and obtain the ...
9
votes
2answers
2k views

Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
41
votes
4answers
114k views

How to calculate a confidence level for a Poisson distribution?

Would like to know how confident I can be in my $\lambda$. Anyone know of a way to set upper and lower confidence levels for a Poisson distribution? Observations ($n$) = 88 Sample mean ($\lambda$) =...
19
votes
1answer
10k views

Should confidence intervals for linear regression coefficients be based on the normal or $t$ distribution?

Let's have some linear model, for example just simple ANOVA: ...
15
votes
1answer
3k views

Prediction interval for binomial random variable

What is the formula (approximate or exact) for a prediction interval for a Binomial random variable? Assume $Y \sim \mathsf{Binom}(n, p)$, and we observe $y$ (drawn from $Y$). The $n$ is known. Our ...
43
votes
3answers
28k views

Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?

Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
30
votes
6answers
14k views

Binomial confidence interval estimation - why is it not symmetric?

I have used the following r code to estimate the confidence intervals of a binomial proportion because I understand that that substitutes for a "power calculation" when designing receiver operating ...
36
votes
6answers
5k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
28
votes
4answers
26k views

How to compute the confidence interval of the ratio of two normal means

I want to derive the limits for the $100(1-\alpha)\%$ confidence interval for the ratio of two means. Suppose, $X_1 \sim N(\theta_1, \sigma^2)$ and $X_2 \sim N(\theta_2, \sigma^2)$ being independent, ...
21
votes
3answers
22k views

Confidence interval of RMSE

I have taken a sample of $n$ data points from a population. Each of these points has a true value (known from ground truth) and an estimated value. I then calculate the error for each sampled point ...
19
votes
2answers
23k views

How to obtain a confidence interval for a percentile?

I have a bunch of raw data values that are dollar amounts and I want to find a confidence interval for a percentile of that data. Is there a formula for such a confidence interval?
10
votes
1answer
454 views

The basic logic of constructing a confidence interval

Consider a model with a parameter of interest, $\theta$, and its point estimator, $\hat\theta$. For simplicity, assume $\hat\theta\sim N(\theta,\sigma^2/n)$ (in numerous instances this could be ...
8
votes
1answer
1k views

Does invertible monotone transformation of a confidence interval give you a confidence interval (at the same level) in the transformed space?

Suppose $$ (a,b) $$ is a $(1-\alpha)$ level confidence interval for a parameter $\theta$. Suppose $\eta$ is a monotone invertible transformation. Then, is $$ \left (\eta(a), \eta(b) \right ) $$ ...
2
votes
2answers
2k views

Why and how does adding an interaction term affects the confidence interval of a main effect?

I was wondering why, in a regression model including an interaction term, the confidence interval of one of the main effects becomes wider. Consider this Cox regression, where the variable IR_BMI27 ...
57
votes
4answers
7k views

Are all values within a 95% confidence interval equally likely?

I have found discordant information on the question: "If one constructs a 95% confidence interval (CI) of a difference in means or a difference in proportions, are all values within the CI equally ...
11
votes
3answers
6k views

What does a confidence interval (vs. a credible interval) actually express? [duplicate]

Possible Duplicate: What, precisely, is a confidence interval? Yes, similar questions have been asked before, but many of the answers seem contradictory and don't address my issue. (Or my ...
11
votes
2answers
3k views

Does it make sense to compute confidence intervals and to test hypotheses when data from whole population is available?

Does it make sense to compute confidence intervals and to test hypotheses when data from the whole population is available? In my opinion, the answer is no, since we can accurately compute the true ...
21
votes
1answer
13k views

Computing prediction intervals for logistic regression

I would like to understand how to generate prediction intervals for logistic regression estimates. I was advised to follow the procedures in Collett's Modelling Binary Data, 2nd Ed p.98-99. After ...
55
votes
6answers
67k views

How to determine best cutoff point and its confidence interval using ROC curve in R?

I have the data of a test that could be used to distinguish normal and tumor cells. According to ROC curve it looks good for this purpose (area under curve is 0.9): My questions are: How to ...
42
votes
5answers
32k views

Confidence interval for median

I have to find a 95% C.I. on the median and other percentiles. I don't know how to approach this. I mainly use R as a programming tool.
34
votes
3answers
5k views

How to find confidence intervals for ratings?

Evan Miller's "How Not to Sort by Average Rating" proposes using the lower bound of a confidence interval to get a sensible aggregate "score" for rated items. However, it's working with a Bernoulli ...
34
votes
2answers
3k views

Is there a reliable nonparametric confidence interval for the mean of a skewed distribution?

Very skewed distributions such as the log-normal do not result in accurate bootstrap confidence intervals. Here is an example showing that the left and right tail areas are far from the ideal 0.025 ...
27
votes
1answer
13k views

Constructing confidence intervals based on profile likelihood

In my elementary statistics course, I learnt how to construct 95% confidence interval such as population mean, $\mu$, based on asymptotic normality for "large" sample sizes. Apart from resampling ...
18
votes
1answer
10k views

Bootstrap-based confidence interval

While studying bootstrap-based confidence interval, I once read the following statement: If the bootstrap distribution is skewed to the right, the bootstrap-based confidence interval incorporates a ...
10
votes
2answers
3k views

Meaning of 2.04 standard errors? Significantly different means when confidence intervals widely overlap?

The image below is from this article in Psychological Science. A colleague pointed out two unusual things about it: According to the caption, the error bars show "±2.04 standard errors, the 95% ...
65
votes
6answers
56k views

Do the predictions of a Random Forest model have a prediction interval?

If I run a randomForest model, I can then make predictions based on the model. Is there a way to get a prediction interval of each of the predictions such that I ...
34
votes
4answers
12k views

Are smaller p-values more convincing?

I've been reading up on $p$-values, type 1 error rates, significance levels, power calculations, effect sizes and the Fisher vs Neyman-Pearson debate. This has left me feeling a bit overwhelmed. I ...
20
votes
3answers
19k views

How do I calculate a confidence interval for the mean of a log-normal data set?

I've heard/seen in several places that you can transform the data set into something that is normal-distributed by taking the logarithm of each sample, calculate the confidence interval for the ...

1
2 3 4 5
12