Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

A confidence interval is an interval that covers an unknown parameter with $(1-\alpha)\%$ confidence. Confidence intervals are a frequentist concept. They are often confused with credible intervals which is the Bayesian analog.

0
votes
0answers
7 views

Boosting precision of confidence intervals using independently obtained confidence intervals

I am asking for the setting in which $\mu_X$, the expectation of a non-negative random variable $X\in\mathbb{N}$, is unknown but via a black box I'm able to obtain an estimate $\hat{\mu}_X$ and ...
-2
votes
0answers
11 views

Intuitive explanation for t vs z in confidence intervals

WAIT DON'T LEAVE YET --- Skip to the conclusion if TL;DR What is an intuitive explanation for what a t-score gives you when computing confidence intervals? Background of my understanding of ...
0
votes
0answers
17 views

Confidence interval for one sided hypothesis tests

I am working on a drug trial where the sponsor wants to have a one sided 10% hypothesis test on the primary outcome. As a standard, we present both p-values and confidence intervals. What will be ...
1
vote
0answers
17 views

Computing confidence intervals for effect sizes from a contingency table

I am studying the relationship between two categorical variables, $X_1$ and $X_2$. Per observation, $X_1$ can take on 1 of 4 possible values, whereas $X_2$ can take on 1 of 3. To test the ...
0
votes
0answers
15 views

Parameter significance: overlapping Confidence Intervals vs. Standard Errors

Assume that we have a graphical representation of some parameter estimates, where the mean values are plotted with additional bars representing either confidential intervals (CI) or standard errors (...
0
votes
0answers
34 views

Calculation of the coverage of variable selection methods

I'm having a problem with understanding the coverage calculation for some variable selection method. Suppose that we want to estimate the coverage for some regression coefficient $\beta_1$ of ...
0
votes
0answers
24 views

Bonferroni correction for pairwise proportion comparisons for more than one outcome level

Suppose I have an event that can have 3 different outcomes. This event is recorded for 21 different factor levels for a total of 100 times, and each time an outcome happens it is counted (so I am ...
0
votes
0answers
7 views

Confidence Intervals for N trials each of sample size n [closed]

How do we create confidence intervals from 2 dice rolls. Lets say there are N trials & each trial is of sample size n=60Million. How do I convert this problem into a confidence interval across the ...
0
votes
0answers
18 views

Epsilon Greedy Performing better than UCB

I am implementing the bandit problem using various algorithms. The issue that i am facing is that epsilon greedy is performing better than UCB for 5arms and horizon of 2000 for epsilon value of 0.95. ...
1
vote
0answers
19 views

Overconfidence of Bayesian classifiers on out-of-distribution samples

I am trying to find a principled way around the following problem: consider a Bayesian classifier $p(x|c)$ where $x$ is the input and $c$ is the class label. According to Bayes rule the class ...
3
votes
0answers
33 views

Confidence interval of proportion of a proportion

Let's say we have $N$ users. Some of them could be hackers and some not. Only for a random $p_u$ percent (e.g. 1%) of users have we actually investigated manually to know which ones are hackers and ...
-1
votes
0answers
10 views

Is the confidence level of the optional answers in a survey reliable as required ones?

Is the confidence level of the optional answers in a survey as reliable as required ones? I have a survey of 250 respondents. For one required question, 15 people answered “I don’t want to play this ...
3
votes
0answers
25 views

Meta-analysis using bootstrapped confidence intervals (non-normal data)

I have the results of two studies (my own and a previous study). I want to perform a small meta-analysis to obtain a better estimate of the effect size. For the two studies I have the means and 95% ...
-1
votes
2answers
50 views

Confidence Intervals meaning [duplicate]

Lets say I know population std. deviation $\sigma=5$ and sample size $n=60$. Now lets say my sample mean is $\overline{X}=98$. I already know from central limit theorem that 95% of the time the sample ...
3
votes
1answer
24 views

Choosing conf.type for survfit in R

I'm curious to learn more about choosing the confidence type within the survfit function in R. I know the options are ...
0
votes
0answers
29 views

Is Tom Mitchell's Machine Learning book incorrect in its definition of Confidence Interval? [duplicate]

In Tom Mitchell's 1997 Machine Learning, a confidence interval is defined on page 138 as An N% confidence interval for some parameter p is an interval that is expected with probability N% to ...
0
votes
0answers
36 views

Estimate the mean of a process

I did my analysis on a data and in the end, I came up with values (6 values). Each one of these values describes how much the phenomena that I'm trying to measure is related at a certain point in time....
0
votes
0answers
26 views

Confidence interval for difference of tail errors

Suppose, that I'm conducting a simulation study that compares different confidence interval construction methods. If I do 1000 simulation runs, then the acceptable range for coverage level is $95\pm\...
0
votes
1answer
15 views

Confidence interval for the mean of a bounded non stationary random variableb given N samples

I have a random variable bounded on [0,1]. The mean can be modeled as a random process. Assume the change of the mean |mu_i - mu_{i+1}| is bounded with a known bound. Given N (N is not large) samples (...
1
vote
1answer
45 views

Invert P-Value to get Confidence Interval

This is a weird problem, but please bear with me. I could use some insight. Imagine you have a black box that spits out p-values for $H_0: \theta = 0$. We'll call this black box $g(\cdot)$. Assume ...
0
votes
0answers
3 views

Manual calculation of confidence intervals for population-attributable fraction in the presence of confounding

Note: I do not how to format appropriately the below mathematical notation. Please feel free to edit to enhance readability. In its simplest form, the population-attributable fraction (attributable ...
0
votes
0answers
10 views

On the confidence intervals of matrix autocorrelation

Considering the case of signal autocorrelation in 1D, I've read that the distribution of the correlation coefficients $r$ of a white noise (independent and identically distributed random variable) ...
0
votes
0answers
15 views

Calculate model accuracy confidence intervals from caret model object?

I am trying to generate 95% confidence intervals for the accuracy predictions on the trained data set using the caret package and interface. Some dummy code: ...
1
vote
1answer
32 views

Highest Posterior Density (HPD) region of the marginals vs. of the joint distribution

In a Bayesian context, to analyse the posterior distribution, one can define the Highest Posterior Density (HPD) region or interval as $$\{\theta; \pi(\theta \mid x) \geq k\} $$ in both unidimensional ...
1
vote
0answers
15 views

How to calculate confidence bounds for this median unbiased estimate

I am aiming to calculate confidence bounds for the median unbiased estimates, retrieved via the procedure described in Stock and Watson (1998). The authors that I am following state the following ...
0
votes
0answers
12 views

AB testing for low traffic sites

I am working on a AB test on a specific page that has very low traffic (like ~1200 per day). As most of us know, features with smaller traffic need more time to detect any significant change unless ...
0
votes
0answers
16 views

Convergence of weighting function

Can someone please explain how $w(\alpha, 1+ \frac{1}{y}) \to \frac{\alpha^{-y}}{\sqrt{y}}$ as $y \to \infty$? I know that this is equivalent to $$\lim_{y \to \infty} \frac{t_{1/y,\alpha/2}}{1+\frac{1}...
1
vote
0answers
10 views

Confidence interval from weighted SD [closed]

How do I calculateconfidence interval from weighted standard deviation? Is it by usual way of calculating CI or there is any separate way? Thank you
1
vote
3answers
63 views

Confidence in precision in the presence of few positives

I would like to report precision and recall for an existing binary classifier, which is a black box and which I cannot modify. I have a 1,000 test examples, sampled from a population of 100,000, and ...
3
votes
1answer
74 views

Does confidence interval for odds ratio assume log-normal distribution?

The formula floating around for calculating 95% confidence interval of an odds ratio is: e^(log(OR) ± 1.96 x sqrt(1/a + 1/b + 1/c + 1/d)) Does one need to ...
2
votes
0answers
16 views

Complement of a confidence set: standard name?

In the context of statistical inference, is there a standard name for the complement of a confidence set? For example, suppose $X$ captures the observed data, $\theta$ the parameter of interest, $\...
0
votes
0answers
31 views

Confidence interval for a macro/micro average

I'm working on a supervised multi class classifier that labels texts according to three possible classes. I calculated the one-vs-all precision, recall and F1 score for each class and the macro ...
1
vote
2answers
47 views

confidence interval of the difference of two statistics

I have the returns of two groups of stock over 135 months and I am trying to find the confidence interval for the difference of two Sharpe ratios (that I will call $X$ and $Y$). To do so, I want to ...
1
vote
1answer
25 views

Calculating effect size + 95 CIs for median differences (SPSS)

I've got a mixture of parametric and non-parametric data, with a repeated measures design; measuring animal behaviour in three different treatments. For parametric data, it's simple enough calculating ...
1
vote
1answer
33 views

Confidence interval for linear models

I have been using a regplot tool from the seaborn recently, and I really liked its plots where it shows both the regression line,...
5
votes
2answers
93 views

Confidence interval for the mean - Normal distribution or Student's t-distribution?

I have taken $N$ measurements $x_i$ of the time taken by a computer program to run a particular calculation, and would like to calculate a 95% confidence interval for the mean runtime. We can ...
0
votes
1answer
41 views

Which type of distribution is associated with a continuous variable that can take values between zero and a constant?

I'm working in descriptive statistics of a variable that is the duration of an animal's response given an 8 s interval window. Thus, this variable is bounded between 0 and 8 s. I want to estimate ...
1
vote
2answers
51 views

What is the difference between bootstrap hypothesis testing/permutation test and traditional hypothesis testing?

I recently learned about doing hypothesis tests by bootstrapping and permutation. I'm trying to relate it to other traditional hypothesis testing such as t-test or chi-square test. Are they in the ...
1
vote
1answer
22 views

How can I estimate uncertainty for a small sample of proportion data?

I'm not an expert on this topic. I'm working on a poster for a conference. I need to plot SEMs, confidence intervals, or some measure of uncertainty around a measure of central tendency (e.g., median)....
3
votes
1answer
20 views

How to estimate the sum of N observations taking from a normal distribution

I would like to get 95% confidence interval for the sum of N observation taken from a normal distribution. I know how to build a confidence interval for a single observation but not for the sum of ...
5
votes
1answer
133 views

Can confidence interval of positive values be negative?

Can confidence interval of positive values be negative if I use sample mean +- sample standard deviation? If I need a log-log plot (with confidence interval), what ...
4
votes
3answers
47 views

Bootstrap confidence intervals - how many replications to choose?

I applied a bootstrap-process to calculate confidence intervalls for the paramters of a multiple lineare regression. In R it's pretty simple to implement (functions: 'boot' and 'boot.ci') but I still ...
0
votes
0answers
13 views

Determining the correct sample size formula

I need to extract a sample from a known population. Each element of this population was observed in the exaxct interval of time. Once this sample is extracted, I will run a time series regression on ...
1
vote
0answers
21 views

confidence intervals for GLMER model

I'm trying to calculate confidence intervals for GLMER model. I heard that there are two ways I could do it: Number 1: exp(Estimate Std + 1.96 * Std Error) exp(ES - 1.96* Std Error) Number 2: m&...
22
votes
3answers
520 views

Is it true that the percentile bootstrap should never be used?

In the MIT OpenCourseWare notes for 18.05 Introduction to Probability and Statistics, Spring 2014 (currently available here), it states: The bootstrap percentile method is appealing due to its ...
0
votes
1answer
13 views

Number of samples required to estimate a desired False Positive Rate

I have an algorithm that for each sample $x_i$ returns an anomaly score $0<s_i<1$. I use cross validation to set a threshold $th$ such that $x_i$ is anomalous if $s_i>th$. During cross ...
0
votes
0answers
12 views

Statistical Analysis on Yearly data

I have some data for the total yearly production of a product spanning over five years (essentially five numbers). I want to create a confidence interval to estimate the high and low of production ...
2
votes
1answer
120 views

Discrepancies in the standard errors calculated in different ways

I noticed discrepancies in the standard errors (SE) calculated in different ways and I would like to know the reason for it. This question is a follow-up of this one. Here is an example based on a ...
0
votes
0answers
22 views
24
votes
5answers
2k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...