Where is my potential flaw in my Z-test of Proportions?

Question

I was working through this z-test of proportions example I found online. The online example solutions says that the difference between the groups is statistically significant, whereas I concluded it was not.

Problem:

Suppose the Acme Drug Company develops a new drug, designed to prevent colds. The company states that the drug is equally effective for men and women. To test this claim, they choose a a simple random sample of 100 women and 200 men from a population of 100,000 volunteers.

At the end of the study, 38% of the women caught a cold; and 51% of the men caught a cold. Based on these findings, can we reject the company's claim that the drug is equally effective for men and women? Use a 0.05 level of significance.

My approach to the problem:

H₀ $\rightarrow$ difference in groups = 0
H₁ $\rightarrow$ difference in groups $\neq$ 0

Goal: Get a 95% Confidence Interval around the difference in groups metric. See if 0 in is the interval. If not, we can reject the null hypothesis.

Difference in groups = $\hat d$ = .51 - .38 = .13

95% C.I. formula = .13 $\pm$ (Z-value for two-tailed test with .05 significance) * (Standard Error)

Standard Error for Z-test of proportions = SE = $\sqrt {p_{pooled}(1-p_{pooled})(\frac{1}{100}+\frac{1}{200})}$

$p_{pooled}$ = (102+38)/(300) = .46667

SE = $\sqrt {.46667(.53333)(.015)}$ = .06107

Z-value for two-tailed test with .05 significance = 1.96

95% C.I. formula = .13 $\pm$ (1.96) * (.06107) = (-0.0103, .2495)

Since 0 is in the 95% C.I. formula, we cannot reject the null that H₀ is equal to zero.

However, the source uses a different methodology and arrives at a differing conclusion:

Solution: The solution to this problem takes four steps: (1) state the hypotheses, (2) formulate an analysis plan, (3) analyze sample data, and (4) interpret results. We work through those steps below:

State the hypotheses. The first step is to state the null hypothesis and an alternative hypothesis. Null hypothesis: P1 = P2

Alternative hypothesis: P1 ≠ P2

Note that these hypotheses constitute a two-tailed test. The null hypothesis will be rejected if the proportion from population 1 is too big or if it is too small. Formulate an analysis plan. For this analysis, the significance level is 0.05. The test method is a two-proportion z-test. Analyze sample data. Using sample data, we calculate the pooled sample proportion (p) and the standard error (SE). Using those measures, we compute the z-score test statistic (z). p = (p1 * n1 + p2 * n2) / (n1 + n2)

p = [(0.38 * 100) + (0.51 * 200)] / (100 + 200)

p = 140/300 = 0.467

SE = sqrt{ p * ( 1 - p ) * [ (1/n1) + (1/n2) ] }

SE = sqrt [ 0.467 * 0.533 * ( 1/100 + 1/200 ) ]

SE = sqrt [0.003733] = 0.061

z = (p1 - p2) / SE = (0.38 - 0.51)/0.061 = -2.13

where p1 is the sample proportion in sample 1, where p2 is the sample proportion in sample 2, n1 is the size of sample 1, and n2 is the size of sample 2.

Since we have a two-tailed test, the P-value is the probability that the z-score is less than -2.13 or greater than 2.13.

We use the Normal Distribution Calculator to find P(z < -2.13) = 0.017, and P(z > 2.13) = 0.017. Thus, the P-value = 0.017 + 0.017 = 0.034. Interpret results. Since the P-value (0.034) is less than the significance level (0.05), we cannot accept the null hypothesis.

Where did I go wrong in my approach?

Answer 1

Your CI calculation was not correctly carried out: $0.13 - 1.96 \times 0.06107$ is positive, not negative.

A sensible first step when you disagree with a solution in this way is to carefully check such things; simple mistakes (such as a typo on a calculator) are often the first thing to try to rule out.

[However, one thing to keep in mind is that both the test and the confidence interval rely on an asymptotic approximation, and the test is being carried out assuming the proportions are equal while the confidence interval would more correctly assume they're unequal; in that case the asymptotic interval and the asymptotic test may not exactly correspond. This doesn't relate to the cause of your mismatch with the solution because your interval assumed the null when calculating the standard error.]