Chi-square test or Z-test for comparing two proportions?

Question

On this website the appropriate statistical test for comparing two independent proportions is described as a $Z$-test (a normal distribution is used to obtain a $p$-value).

However, in R prop.test, a Chi-square distribution is invoked to obtain the $p$-value comparing two independent proportions.

An R example is:

prop.test(x = c(474, 376), n = c(750, 750))

I was wondering how a Chi-square based approach comparing two independent proportions works (e.g., is the test a chi-square test or a different test that produces a statistic that follows a Chi-square distribution) and how it compares with the Z-test approach (e.g., in terms of accuracy and power) discussed on that website?

Answer 1

Edit, 2024:

I'm retracting most of my previous answer, which was written about seven years ago. I can't delete it, since it is the accepted answer.

I think the answer by @qwr answers the heart of the question.

I do think, though, that in the R environment, thinking of the test as a chi-square test may be more beneficial as it allows for more options in terms of continuity correction or Monte Carlo approaches.

Here, I expanded the code by @qwr, to make it a little more user-friendly. For either a small sample or large sample, the Z-score method and chi-square method return the same result.

bin_test <- function(x1, n1, x2, n2) {
  p1 <- x1 / n1
  p2 <- x2 / n2
  p <- (x1 + x2) / (n1 + n2)
  z <- (p2 - p1) / sqrt(p*(1-p)*(1/n1 + 1/n2))
  z
}


X = c(1, 2)   ### Numerators

D = c(10, 10) ### Denominators

Zsquare = bin_test(X[1], D[1], X[2], D[2])^2

Zsquare

Z = sqrt(Zsquare)

Z

pnorm(Z, lower.tail=FALSE) * 2

prop.test(x = X, n = D, correct=FALSE)

Original answer follows. Please disregard.

Caveat: This is my non-statistician view of the question.

The following website discusses the two approaches. The prop.test approach is the statistically fundamental one. The z-test approach is, in my eyes, an approximation of this, and relies on certain assumptions about the data.

www.r-bloggers.com/comparison-of-two-proportions-parametric-z-test-and-non-parametric-chi-squared-methods/

Since R has good support for tests of proportions (confidence intervals, binom.test, multinomial.test), I don't see why one would want to use the z-test approach. But that may just be my bias and particular education.

Considering using confidence intervals in R, the following two pages have examples of confidence intervals for proportions, binom.test, and multinomial.test in R. (Caveat: I am the author of these two webpages.)

rcompanion.org/handbook/H_03.html

rcompanion.org/handbook/H_02.html

Answer 2

The tests are equivalent. They both test the same null hypothesis $H_0: p_1 = p_2$, versus $H_1: p_1 \ne p_2$ (for the two-sided case).

In the two-sample binomial test, we assume a Normal approximation and test the statistic $$Z = \frac{\hat p_1 - \hat p_2}{\sqrt{\hat p (1 - \hat p)( 1 /n_1 + 1 / n_2 )}} \sim N(0, 1)$$

The denominator is the pooled standard error, with $p$ estimated by the weighted average of $\hat p_1$ and $\hat p_2$:
$$\hat p = \frac{n_1 \hat p_1 + n_2 \hat p_2}{n_1 + n_2} = \frac{x_1 + x_2}{n_1 + n_2}$$

In the 2x2 contingency table method, we again assume the Normal approximation and test the statistic $$X^2 = \sum_{i=1}^2 \sum_{j=1}^2 \frac{(O_{ij} - E_{ij})^2}{E_{ij}} \sim \chi^2_1$$

The mathematical justification for the chi-square distribution is based on the idea of summing "z-scores" of each cell's observed versus expected count.

The resulting p-values should be identical, since by definition $\chi^2_1 = Z^2$.

Ref. Section 10.2 of Fundamentals of Biostatistics, 3ed (1990) by Bernard Rosner.

---

Before I knew about prop.test(), I implemented the two-sample pooled-variance binomial (approximation) z-test from NIST Engineering Statistics Handbook. This uses the pooled standard error $\sqrt{\hat p (1 - \hat p) ( 1 / n_1 + 1 / n_2 ) } $, since under the null hypothesis, $p_1 \sim N(p, p(1-p)/n_1)$ and $p_2 \sim N(p, p(1-p)/n_2)$, and $p$ is estimated by the weighted average $\hat p$ given above.

> bin_test <- function(x1, n1, x2, n2) {
  p1 <- x1 / n1
  p2 <- x2 / n2
  p <- (x1 + x2) / (n1 + n2)
  z <- (p2 - p1) / sqrt(p*(1-p)*(1/n1 + 1/n2))
  z
}

> bin_test(474, 750, 376, 750)^2
[1] 26.07421

> prop.test(x = c(474, 376), n = c(750, 750))

    2-sample test for equality of proportions with continuity correction

data:  c(474, 376) out of c(750, 750)
X-squared = 25.545, df = 1, p-value = 4.322e-07
alternative hypothesis: two.sided
95 percent confidence interval:
 0.07961695 0.18171638
sample estimates:
   prop 1    prop 2 
0.6320000 0.5013333 

The statistic is close, but slightly more conservative in p-value. The R docs says Yates's continuity correction is applied by default. If you turn that off, the result is exact:

> prop.test(x = c(474, 376), n = c(750, 750), correct=FALSE)

    2-sample test for equality of proportions without continuity correction

data:  c(474, 376) out of c(750, 750)
X-squared = 26.074, df = 1, p-value = 3.285e-07
alternative hypothesis: two.sided
95 percent confidence interval:
 0.08095028 0.18038305
sample estimates:
   prop 1    prop 2 
0.6320000 0.5013333