Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In the excellent Practical Statistics for Medical Research Douglas Altman writes in page 235: "Because the standard error used for calculating the confidence interval differs from that used in hypothesis testing it can occasionally happen[...] that the confidence interval excludes the value specified under the null hypothesis when the hypothesis gives a non-significant result"

Could someone comment why the SE's are different in hypothesis testing than in confidence intervals construction and which formulas are appropriate in each case?

share|improve this question
Could this be due to Wald vs. other types of test? I have seen this happen in SAS for logistic regression, where they use different default methods for CI and p. What is the context around p. 235 where he writes this? – Peter Flom Mar 5 '13 at 12:08
It's under 10.3 Proportions in two independent groups with 10.3.1 Confidence interval , 10.3.2 Hypothesis test and 10.3.3 Continuity correction. The passage appears as a general conclusion of 10.3. Chapter 10 is about Comparing groups - categorical data – nostock Mar 5 '13 at 15:44

One simple example of this is doing a one sample test of proportions using the normal approximation. When doing a test of significance we have a null hypothesis that the proportion is a specific value, so we use that number in the standard error formula (since we do not know the true proportion and assume the null is true till proven otherwise). But when doing a confidence interval we do not assume anything about the proportion and generally use the proportion estimated from the sample in the standard error formula. Occassionally this can make the 2 disagree.

Similarly when comparing 2 proportions the standard error in hypothesis testing uses a composite proportion assuming that the 2 proportions are equal, but the confidence interval does not assume equality and combines the 2 proportions in a different way.

share|improve this answer
Is there some reference list for the different formulas? – nostock Mar 6 '13 at 7:41
@nostock, most introductory stats textbooks will show these formulas. – Greg Snow Mar 7 '13 at 1:03
You will be surprised how many they don't and how many do not clearly point these differences. – nostock Mar 8 '13 at 7:16

The above link has some detailed explanation on why we calculate SE separately for CIs and Hypothesis testing of proportions..

share|improve this answer
Hello, welcome to CV! Answers are better when they are standalone, meaning that a reader can understand the answer without referring to an external link. (Also, external pages can change or move.) Would you mind expanding your answer a little bit? – Patrick Coulombe Jul 22 '14 at 6:24
While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – QuantIbex Jul 22 '14 at 8:59
The target of that link provides the same explanation as Greg Snow's answer. – whuber Jul 22 '14 at 12:59

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.