Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In the excellent Practical Statistics for Medical Research Douglas Altman writes in page 235: "Because the standard error used for calculating the confidence interval differs from that used in hypothesis testing it can occasionally happen[...] that the confidence interval excludes the value specified under the null hypothesis when the hypothesis gives a non-significant result"

Could someone comment why the SE's are different in hypothesis testing than in confidence intervals construction and which formulas are appropriate in each case?

share|improve this question
    
Could this be due to Wald vs. other types of test? I have seen this happen in SAS for logistic regression, where they use different default methods for CI and p. What is the context around p. 235 where he writes this? –  Peter Flom Mar 5 '13 at 12:08
1  
It's under 10.3 Proportions in two independent groups with 10.3.1 Confidence interval , 10.3.2 Hypothesis test and 10.3.3 Continuity correction. The passage appears as a general conclusion of 10.3. Chapter 10 is about Comparing groups - categorical data –  nostock Mar 5 '13 at 15:44
add comment

1 Answer

One simple example of this is doing a one sample test of proportions using the normal approximation. When doing a test of significance we have a null hypothesis that the proportion is a specific value, so we use that number in the standard error formula (since we do not know the true proportion and assume the null is true till proven otherwise). But when doing a confidence interval we do not assume anything about the proportion and generally use the proportion estimated from the sample in the standard error formula. Occassionally this can make the 2 disagree.

Similarly when comparing 2 proportions the standard error in hypothesis testing uses a composite proportion assuming that the 2 proportions are equal, but the confidence interval does not assume equality and combines the 2 proportions in a different way.

share|improve this answer
    
Is there some reference list for the different formulas? –  nostock Mar 6 '13 at 7:41
    
@nostock, most introductory stats textbooks will show these formulas. –  Greg Snow Mar 7 '13 at 1:03
    
You will be surprised how many they don't and how many do not clearly point these differences. –  nostock Mar 8 '13 at 7:16
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.