# Both Z-test and chi-squared apply in testing if a pill makes any difference in avoiding the flu?

I want to test if a certain pill would make any difference in avoiding the flu.

I give the pill to group A and a placebo to group B. Then I count how many individuals got the flu in each group, and how many didn't.

I imagine two ways of testing if the pill made any difference in avoiding the flu:

Z test for proportion: I conduct a z test with the following hypoteshis: Null hypothesis: The proportion of people that got the flu in group A is equal to the proportion of people that got the flu in group B. Alternative Hypothesis: the proportions are different.

Chi-squared test: I create a contingency table with the sample and I conduct a chi-squared test for homogeneity.

What's the most adequate test in this case, and why?

• related to this question Feb 4 '21 at 14:46

There are various versions of both tests. If the error variance in the test of proportions is estimated from the two groups separately, and no continuity correction is used for either test, then the two tests may be equivalent.

Suppose there were 403 successes out of 500 subjects in the drug group and 366 out of 500 in the placebo group, then here are the two tests as computed in R. Notice that the methods of formatting the data and the methods of formatting the output differ between the two tests as implemented in R.

Test of binomial proportions. Data are numbers of successes and sample sizes. The continuity correction is declined (parameter cor=F).

prop.test(c(403,366), c(500,500), cor=F)

2-sample test for equality of proportions
without continuity correction

data:  c(403, 366) out of c(500, 500)
X-squared = 7.7066, df = 1, p-value = 0.005502
alternative hypothesis: two.sided
95 percent confidence interval:
0.0219564 0.1260436
sample estimates:
prop 1 prop 2
0.806  0.732


Chi-squared test. Data are provided in a contingency table. The Yates correction is declined.

TBL
[,1] [,2]
[1,]  403  366
[2,]   97  134

chisq.test(TBL, cor=F)

Pearson's Chi-squared test

data:  TBL
X-squared = 7.7066, df = 1, p-value = 0.005502


Notes: (1) Proportions test: If a pooled error variance is used in the test of binomial proportions, then the confidence intervals do not exactly match the significance test. That is, 95% confidence intervals excluding $$0$$ do not exactly match rejection at the 5% level. Also, the X-squared statistic shown in the R output for this test is the square of the z-statistic, shown by some other statistical software. Relevant Minitab output below:

Test and CI for Two Proportions

Sample    X    N  Sample p
1       403  500  0.806000
2       366  500  0.732000

Difference = p (1) - p (2)
Estimate for difference:  0.074
95% CI for difference:  (0.0219564, 0.126044)
Test for difference = 0 (vs ≠ 0):  Z = 2.79  P-Value = 0.005

Fisher’s exact test: P-Value = 0.007


(2) Chi-squared test: If counts are sufficiently small for the (somewhat overly conservative) Yates correction to make a difference whether to reject, then perhaps it is best to use a Fisher exact test instead. The P-value of Fisher's test for my fake data is shown below.

fisher.test(TBL)\$p.val
[1] 0.006834361