5
$\begingroup$

In the excellent Practical Statistics for Medical Research Douglas Altman writes in page 235: "Because the standard error used for calculating the confidence interval differs from that used in hypothesis testing it can occasionally happen[...] that the confidence interval excludes the value specified under the null hypothesis when the hypothesis gives a non-significant result"

Could someone comment why the SE's are different in hypothesis testing than in confidence intervals construction and which formulas are appropriate in each case?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Could this be due to Wald vs. other types of test? I have seen this happen in SAS for logistic regression, where they use different default methods for CI and p. What is the context around p. 235 where he writes this? $\endgroup$
    – Peter Flom
    Mar 5, 2013 at 12:08
  • 1
    $\begingroup$ It's under 10.3 Proportions in two independent groups with 10.3.1 Confidence interval , 10.3.2 Hypothesis test and 10.3.3 Continuity correction. The passage appears as a general conclusion of 10.3. Chapter 10 is about Comparing groups - categorical data $\endgroup$
    – nostock
    Mar 5, 2013 at 15:44

2 Answers 2

Reset to default
5
$\begingroup$

One simple example of this is doing a one sample test of proportions using the normal approximation. When doing a test of significance we have a null hypothesis that the proportion is a specific value, so we use that number in the standard error formula (since we do not know the true proportion and assume the null is true till proven otherwise). But when doing a confidence interval we do not assume anything about the proportion and generally use the proportion estimated from the sample in the standard error formula. Occassionally this can make the 2 disagree.

Similarly when comparing 2 proportions the standard error in hypothesis testing uses a composite proportion assuming that the 2 proportions are equal, but the confidence interval does not assume equality and combines the 2 proportions in a different way.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Is there some reference list for the different formulas? $\endgroup$
    – nostock
    Mar 6, 2013 at 7:41
  • $\begingroup$ @nostock, most introductory stats textbooks will show these formulas. $\endgroup$
    – Greg Snow
    Mar 7, 2013 at 1:03
  • $\begingroup$ You will be surprised how many they don't and how many do not clearly point these differences. $\endgroup$
    – nostock
    Mar 8, 2013 at 7:16
-1
$\begingroup$

http://apcentral.collegeboard.com/apc/members/courses/teachers_corner/49013.html

The above link has some detailed explanation on why we calculate SE separately for CIs and Hypothesis testing of proportions..

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ Hello, welcome to CV! Answers are better when they are standalone, meaning that a reader can understand the answer without referring to an external link. (Also, external pages can change or move.) Would you mind expanding your answer a little bit? $\endgroup$
    – Patrick Coulombe
    Jul 22, 2014 at 6:24
  • 1
    $\begingroup$ The target of that link provides the same explanation as Greg Snow's answer. $\endgroup$
    – whuber ♦
    Jul 22, 2014 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.