6
$\begingroup$

I'm trying to test the difference in proportions using the z test method and chi-squared method, but am getting very different answers. Is that normal?

My data:

        CI     CII
Male    205    102
Female  83     39

Calculating the z score I get 0.25 which should correlate to a p-value of 0.4013. Calculating the chi-squared score I get 0.0626 correlating to a p-value of 0.8025.

I read that the z-score requires some assumptions (probability of success is ~0.5 and n is high). Is this violating those? Or is it just the nature of these different approaches that gives very different answers with the same meaning (no evidence of difference).

I'm certainly open to miscalculations, but I've re-checked. If this behaviour isn't normal I'll recheck again!

Here are my calculations in R.

> r1 <- 205
> r2 <- 102
> n1 <- 288
> n2 <- 141
> (p1 <- r1/n1)
[1] 0.7118056
> (p2 <- r2/n2)
[1] 0.7234043
> (common.proportion <- (r1+r2)/(n1+n2))
[1] 0.7156177
> (se.pooled <- sqrt(common.proportion*(1-common.proportion)*(1/n1+1/n2)))
[1] 0.0463676
> (zscore <- (p1-p2)/se.pooled)
[1] -0.2501466
> 
> # chi-squared
> prop.test(c(205,102), c(288,141), correct = FALSE)

    2-sample test for equality of proportions without continuity
    correction

data:  c(205, 102) out of c(288, 141)
X-squared = 0.0626, df = 1, p-value = 0.8025
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.10208385  0.07888645
sample estimates:
   prop 1    prop 2 
0.7118056 0.7234043 
$\endgroup$
2
  • $\begingroup$ A related issue. If I'm presenting confidence intervals around each proportion calculated using the binomial distribution, but then comparing them and presenting a p-value using chi-squared, that seems a bit wrong. I might have overlapping CIs, but then a p-value that's less than 0.05. Am I thinking correctly? $\endgroup$
    – Tom
    Mar 13, 2015 at 4:29
  • 3
    $\begingroup$ Tom, can you show your math? These two tests should give very similar results for your sample sizes (especially if making the same choice about continuity corrections). $\endgroup$
    – Alexis
    Mar 13, 2015 at 5:19

1 Answer 1

7
$\begingroup$

Very simple: both the z test and the contingency table $\chi^{2}$ test are two tailed tests, but you have got the one-sided $p$-value for your z test statistic. That is for $H_{0}: p_{1} - p_{2} = 0$, the $p$-value = $P(|Z| \ge |z|)$, but your reported $p$-value is only $P(Z \le z)$.

Notice that $0.4013 \times 2 \approx 0.8025$. Easy!

$\endgroup$
3
  • 6
    $\begingroup$ And the square of the Z-score is (−0.2501466)² = 0.06257, which equals the test statistic X-squared from the prop.test() output. $\endgroup$ Mar 13, 2015 at 18:55
  • 1
    $\begingroup$ Thank you for that "easy" answer! (Which prompted a good schooling on one-tailed tests. And now @KarlOveHufthammer you're sending me down another schooling. If x-squared is simply the z score squared, why do we even have it? And why isn't it called z-squared? (Obviously, not to be answered here. I have a lot to learn!) $\endgroup$
    – Tom
    Mar 16, 2015 at 6:29
  • 2
    $\begingroup$ @Tom The reason that it’s called X-squared in the R output, is that the X is really an ASCII interpretation of the greek capital letter chi (Χ) (the lowercase version of this letter looks like this: χ). And it’s a chi-squared test, which is used in lots of other situations. That said, the test(s) should never be used for comparing binomial proportions, as they have terrible statistical properties. See stats.stackexchange.com/questions/82720/… $\endgroup$ Mar 16, 2015 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.