Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have read about controversies regarding hypothesis testing with some commentators suggesting that hypothesis testing should not be used. Some commentators suggest that confidence intervals should be used instead.

  • What is the difference between confidence intervals and hypothesis testing? Explanation with reference and examples would be appreciated.
share|improve this question
4  
I think you wanted to ask why reporting the hypothesis testing results by showing confidence interval is better than just saying something is confirmed or rejected on some p-value level. –  mbq Oct 1 '11 at 9:53
2  
You should consider checking some of your other questions as answered. –  Andy W Oct 1 '11 at 13:13
    
What is the relation between 95% CI and two-tailed testing hypothesis with alfa=0.05? are they same?If not then how? –  love-stats Oct 1 '11 at 13:40

3 Answers 3

up vote 9 down vote accepted

You can use a confidence interval (CI) for hypothesis testing. In the typical case, if the CI for an effect does not span 0 then you can reject the null hypothesis. But a CI can be used for more, whereas reporting whether it has been passed is the limit of the usefulness of a test.

The reason you're recommended to use CI instead of just a t-test, for example, is because then you can do more than just test hypotheses. You can make a statement about the range of effects you believe to be likely (the ones in the CI). You can't do that with just a t-test. You can also use it to make statements about the null, which you can't do with a t-test. If the t-test doesn't reject the null then you just say that you can't reject the null, which isn't saying much. But if you have a narrow confidence interval around the null then you can suggest that the null, or a value close to it, is likely the true value and suggest the effect of the treatment, or independent variable, is too small to be meaningful (or that your experiment doesn't have enough power and precision to detect an effect important to you because the CI includes both that effect and 0).

share|improve this answer
    
One addition to @john 's comments: First, sometimes the key question is whether the CI spans 1, not 0 (e.g. logistic regression). –  Peter Flom Oct 1 '11 at 9:23
    
Guys, it is 1 or is it 0? (This looks very illuminating to me, so I guess, I need to learn the correct value to look out for!) @John –  Adhesh Josh Oct 1 '11 at 12:06
    
What is the relation between 95% CI and two-tailed testing hypothesis with alfa=0.05? are they same?If not then how? –  love-stats Oct 1 '11 at 13:49
    
love-stats, when used the same they are the same. –  John Oct 1 '11 at 14:51
    
Adhesh Josh, the null hypothesis can be any fixed value specified beforehand. That's another feature of the CI over straight NHST. It's very easy to use when you want to test against a hypothetical value other than 0. –  John Oct 1 '11 at 14:53

There is an equivalence between hypothesis tests and confidence intervals. (see e.g. http://en.wikipedia.org/wiki/Confidence_interval#Statistical_hypothesis_testing) I'll give a very specific example. Suppose we have sample $x_1, x_2, ..., x_n$ from a normal distribution with mean $\mu$ and variance 1, which we'll write as $\mathcal N(\mu,1)$. Suppose we think that $\mu = m$, and we want to test the null-hypothesis $H0: \mu = m$, at level 0.05. So we make a test statistic, which in this case we will take to be the sample average: $v = (x_1 + x_2 + ... + x_n ) / n$. Now suppose $A(m)$ is the "acceptance region" for $v$ for this test. That means that $A(m)$ is the set of possible values of $v$ for which the null-hypothesis $\mu=m$ is accepted at level 0.05 (I use "accepted" as a shorthand for "not rejected" -- I am not suggesting that you would conclude the null hypothesis is true.). For this example, we can look at the $\mathcal N(m,1)$ normal distribution and choose any set that has probability at least 0.95 under this distribution. Now, a 95% confidence region for $\mu$ is the set of all $m$ for which $v$ is in $A(m)$. In other words, it is the set of all $m$ for which the null-hypothesis would be accepted for the observed $v$. That's why John says "If the CI for an effect does not span 0 then you can reject the null hypothesis." (John is referring to the case of testing $\mu = 0$.)

A related topic is the p-value. The p-value is the smallest level for a test at which we would reject the null-hypothesis. To tie it in with the discussion of confidence intervals, suppose we get a particular sample average $v$, from which we construct confidence intervals of different sizes. Suppose a 95% confidence interval for $\mu$ does not contain $m$. Then we can reject the null-hypothesis $\mu=m$ at level 0.05. Then suppose we grow the confidence interval until it just touches (but doesn't include) the value $m$, and suppose this is a 98% confidence interval. Then the p-value for the hypothesis $\mu=m$ is 0.02 (which we get from 1-0.98).

share|improve this answer

'Student' argued for confidence intervals on the grounds that they could show which effects were more important as well as which were more significant.

For example, if you found two effects where the first had a confidence interval for its financial impact from £5 to £6, while the second had a confidence interval from £200 to £2800. The first is more statistically significant but the second is probably more important.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.