An interval of random variables, depending on observed data, which, with a fixed probability, contain an unknown parameter of interest.
31
votes
6answers
2k views
Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals
A recent question on the difference between confidence and credible intervals led me to start re-reading Edwin Jaynes' article on that topic:
Jaynes, E. T., 1976. `Confidence Intervals vs Bayesian ...
41
votes
6answers
15k views
What's the difference between a confidence interval and a credible interval?
Joris and Srikant's exchange here got me wondering (again) if my internal explanations for the the difference between confidence intervals and credible intervals were the correct ones. How you would ...
32
votes
2answers
2k views
What are examples where a “naive bootstrap” fails?
Suppose I have a set of sample data from an unknown or complex distribution, and I want to perform some inference on a statistic $T$ of the data. My default inclination is to just generate a bunch of ...
32
votes
7answers
2k views
What, precisely, is a confidence interval?
I know roughly and informally what a confidence interval is. However, I can't seem to wrap my head around one rather important detail: According to Wikipedia:
A confidence interval does not ...
8
votes
2answers
1k views
Confidence interval for Bernoulli sampling
I have a random sample of Bernoulli random variables $X_1 ... X_N$, where $X_i$ are i.i.d. r.v. and $P(X_i = 1) = p$, and $p$ is an unknown parameter.
Obviously, one can find an estimate for $p$: ...
25
votes
10answers
2k views
Clarification on interpreting confidence intervals?
My current understanding of the notion "confidence interval with confidence level $1 - \alpha$" is that if we tried to calculate the confidence interval many times (each time with a fresh sample), it ...
5
votes
1answer
415 views
Discrete functions: Confidence interval coverage?
How to calculate discrete interval coverage?
What I know how to do:
If I had a continuous model, I could define a 95% confidence interval for each of my predicted values, and then see how often the ...
49
votes
6answers
3k views
Why does a 95% CI not imply a 95% chance of containing the mean?
It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
16
votes
2answers
909 views
How to find confidence intervals for ratings?
Evan Miller's "How Not to Sort by Average Rating" proposes using the lower bound of a confidence interval to get a sensible aggregate "score" for rated items. However, it's working with a Bernoulli ...
16
votes
6answers
3k views
Binomial confidence interval estimation - why is it not symmetric?
I have used the following r code to estimate the confidence intervals of a binomial proportion because I understand that that substitutes for a "power calculation" when designing receiver operating ...
4
votes
2answers
231 views
Sample size for a variable number of answers
I have N different possible results (ex. red, yellow, or blue). I go to a population of infinite size and ask a bunch of people a question the answer to which is one of the N options.... How large ...
2
votes
2answers
247 views
Can someone give me details about a particular confidence interval formula?
I'm trying to fully understand the confidence interval formal given on this site:
$$\hat{\mu}\pm z_{1-\alpha/2}\sqrt{\frac{\hat{\mu}(1-\hat{\mu})}{n}}$$
so I can reproduce the same type of intervals ...
6
votes
1answer
1k views
Should confidence intervals for linear regression coefficients be based on the normal or $t$ distribution?
Let's have some linear model, for example just simple ANOVA:
...
2
votes
1answer
1k views
Wide or narrow confidence interval
I have a project in which I am looking at the correlation betwen fortnightly income and the amount saved each fortnight by low income workers.
I am reporting the effect size and the confidence ...
18
votes
6answers
1k views
When are confidence intervals useful?
If I understand correctly a confidence interval of a parameter is an interval constructed by a method which yields intervals containing the true value for a specified proportion of samples. So the ...
8
votes
1answer
1k views
Confidence interval around the ratio of two proportions
I have two proportions (e.g., clickthrough rate (CTR) on a link in a control layout, and CTR on a link in an experimental layout), and I want to calculate a 95% confidence interval around the ratio of ...
17
votes
7answers
10k views
How to interpret confidence interval of the difference in means in one sample T-test?
SPSS provides the output "confidence interval of the difference means." I have read in some places that it means "95 times out of 100, our sample mean difference will be between between these bounds" ...
20
votes
6answers
654 views
What is the connection between credible regions and Bayesian hypothesis tests?
In frequentist statistics, there is a close connection between confidence intervals and tests. Using inference about $\mu$ in the $\rm N(\mu,\sigma^2)$ distribution as an example, the $1-\alpha$ ...
9
votes
3answers
1k views
Relation between confidence interval and testing statistical hypothesis for t-test
It is well known that confidence intervals and testing statistical hypothesis are strongly related. My questions is focused on comparison of means for two groups based on a numerical variable. Let's ...
6
votes
1answer
481 views
How to report asymmetrical confidence intervals of a proportion?
I've calculated the proportion of chicks fledged out of the number of eggs hatched in each year using prop.test() in R. I see that it gives me the proportion ...
4
votes
1answer
524 views
Difference between CI of ANOVA level coefficients vs t-test CI - which one is “correct”?
How do I get the 95% confidence interval for ANOVA level coefficients? For comparison with a constant value, not multiple comparison (like MMC).
I tried to take the coefficient's SE from the model:
...
2
votes
1answer
954 views
Constructing 95% confidence interval based on profile likelihood
In my elementary statistics course, I learnt how to construct 95% confidence interval such as population mean, mu based on asymptotic normality for "large" sample size. Apart from resampling methods, ...
4
votes
2answers
346 views
Meaning of 2.04 standard errors? Significantly different means when confidence intervals widely overlap?
The image below is from this article in Psychological Science. A colleague pointed out two unusual things about it:
According to the caption, the error bars show "±2.04 standard errors, the 95% ...
2
votes
1answer
278 views
Parameters’ uncertainty for small sample size
Suppose we have a small set of numbers (5 to 10 observations), and we’re trying to fit a distribution to this set. Also, we know that all numbers are positive. I tried to fit lognormal, but I’m not ...
2
votes
2answers
575 views
Calculating the confidence interval for simple linear regression coefficient estimates
I have a data set of paired measurements $(x_1,y_1),(x_2,y_2),...,(x_n,y_n)$. I need to fit a linear regression line $y=ax+b$ to this data. Therefore, I have to estimate the parameters $a$ and $b$.
...
2
votes
1answer
457 views
Simultaneous confidence interval for multiple proportions in R
I want to perform in R the analysis "simultaneous confidence interval for multiple proportions", as illustrated in the article of Agresti et al. (2008) "Simultaneous confidence intervals for comparing ...
1
vote
1answer
442 views
Correct terminology for describing relative confidence interval
My team manages a survey that has been reporting for many years as "relative sample error" a range of figures which are in fact the Xs in "+/- X percent" of an estimate where X is chosen to make up a ...
0
votes
1answer
178 views
Confidence intervals and pairwise comparisons
Let us say I have the following pair wise test data:
...
0
votes
0answers
105 views
How to interpret a 95% confidence interval of a proportion? [duplicate]
Possible Duplicate:
How to interpret confidence interval of the difference in means in one sample T-test?
The prevalence of obesity in children to mothers who smoked throughout pregnancy ...
10
votes
3answers
2k views
Confidence interval for median
I have to find a 95% c.i. on the median and other percentiles. I don't know how to approach this. Any help appreciated. I mainly use R as a programming tool.
7
votes
2answers
3k views
How to draw an interaction plot with confidence intervals?
My attempts:
I couldn't get confidence intervals in interaction.plot()
and on the other hand plotmeans() from package 'gplot' ...
6
votes
2answers
906 views
Obtaining and interpreting bootstrapped confidence intervals from hierarchical data
I'm interested in obtaining a bootstrapped confidence interval on quantity X, when this quantity is measured 10 times in each of 10 individuals.
One approach is to obtain the mean per individual, ...
17
votes
3answers
1k views
How to combine confidence intervals for a variance component of a mixed-effects model when using multiple imputation
The logic of multiple imputation (MI) is to impute the missing values not once but several (typically M=5) times, resulting in M completed datasets. The M completed datasets are then analyzed with ...
3
votes
1answer
354 views
Is there a radical difference in how bayesian and frequentist approaches treat nuisance parameters?
The wiki article on credible intervals has the following statement:
credible intervals and confidence intervals treat nuisance parameters in radically different ways.
What is the radical ...
10
votes
3answers
1k views
Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?
Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function ...
8
votes
6answers
683 views
Statistical argument for why 10,000 heads from 20,000 tosses suggests invalid data
Let's say we are repeatedly tossing a fair coin, and we know number of heads and tails should be roughly equal. When we see a result like 10 heads and 10 tails for a total of 20 tosses, we believe the ...
5
votes
1answer
639 views
CI for a difference based on independent CIs
I'm looking to check my logic here.
Say you measure a quantity in group A, and find the mean is 2 and your 95% confidence interval ranges from 1 to 3. Then you measure the same quantity in group B ...
10
votes
2answers
1k views
What would a confidence interval around a predicted value from a mixed effects model mean?
I was looking at this page and noticed the methods for confidence intervals for lme and lmer in R. For those who don't know R, those are functions for generating mixed effects or multi-level models. ...
9
votes
2answers
707 views
Finding precision of Monte Carlo simulation estimate
Background
I am designing a Monte Carlo simulation that combines the outputs of series of models, and I want to be sure that the simulation will allow me to make reasonable claims about the ...
8
votes
2answers
4k views
Difference between confidence intervals and prediction intervals
For a prediction interval in linear regression you still use $\hat{E}[Y|x] = \hat{\beta_0}+\hat{\beta}_{1}x$ to generate the interval. You also use this to generate a confidence interval of ...
6
votes
1answer
2k views
Coverage probabilities of the basic bootstrap confidence Interval
I have the following question for a course I'm working on:
Conduct a Monte Carlo study to estimate the coverage probabilities of
the standard normal bootstrap confidence interval and the basic ...
5
votes
3answers
931 views
Difference between G-test and t-test and which should be used for A/B testing?
The G-Test: http://en.wikipedia.org/wiki/G-test is a way to get quick estimates of a chi squared distribution, and is recommended by the author of this well-known A/B test tutorial: ...
2
votes
2answers
648 views
Understanding confidence interval
I've made a little questionnaire where participants can rate an answer between 1 and 5. I calculated the mean value, the average value and the standard deviation.
Now I was asking myself if it is ...
9
votes
2answers
2k views
How do I calculate a confidence interval for the mean of a log-normal data set?
I've heard/seen in several places that you can transform the data set into something that is normal-distributed by taking the logarithm of each sample, calculate the confidence interval for the ...
5
votes
3answers
7k views
What is the difference between confidence intervals and hypothesis testing?
I have read about controversies regarding hypothesis testing with some commentators suggesting that hypothesis testing should not be used. Some commentators suggest that confidence intervals should ...
4
votes
1answer
821 views
Computing prediction intervals for logistic regression
I would like to understand how to generate prediction intervals for logistic regression estimates.
I was advised to follow the procedures in Collett's Modelling Binary Data, 2nd Ed p.98-99. After ...
8
votes
3answers
1k views
How can I (numerically) approximate values for a beta distribution with large alpha & beta
Is there a numerically stable way to calculate values of a beta distribution for large integer alpha, beta (e.g. alpha,beta > 1000000)?
Actually, I only need a 99% confidence interval around the ...
7
votes
3answers
2k views
Combining two confidence intervals/point estimates
Suppose one has two independent samples from the same population, and different methods were used on the two samples to derive point estimate and confidence intervals. In trivial cases a sensible ...
6
votes
2answers
926 views
Yates' continuity correction in confidence interval returned by prop.test
This is the confidence interval estimated by prop.test
...
4
votes
2answers
2k views
What is the relationship between a p-value and a confidence interval?
Following from this question on the difference between confidence intervals and hypothesis testing, I would love to have a simple example to better understand the the relationship between confidence ...
