Does a 2*2 chi-square test give the same results as a binomial test?

This is a conceptual question. Suppose we have an experimental design with two grouping variables (e.g., the subjects are either male or female) and the variable we are measuring is a binary variable, like whether they are smokers or not - and we are interested in seeing whether the prevalence of smoking depends on gender.

I feel like both a binomial test and a 2*2 chi-square test would work in this case. For example, we could look at the proportion of yes responses in women and in men, and then compare whether one proportion was significantly different from the other (i.e., the binomial test). Alternatively, we could have a 2*2 contingency table with male/female and smoker/non-smoker and we could do a chi-square test.

My question is, would these two tests give the same results?

What would be the principled way of deciding which test to use in this situation?

• Can you be more explicit about the binomial test you intend? For some of the things you might mean the answer to the title question may be 'yes', for others, 'no' – Glen_b Aug 6 '19 at 22:52

1 Answer

The basic concept is that they should be the same. However, different statistical programs use slight variations that give different answers. Several variations are illustrated here. I have chosen data for which there is no significant effect. (In my experience, P-values vary less in the case of significant effects.)

My fake data are as follows:

Gender\Smoke  Y   N   Tot
F            60  40   100
M           130  70   200


Output from R: Chi-squared test with Yates continuity correction.

a = c(60,40); b = c(130,70)
DTA = cbind(a,b); DTA
a   b
[1,] 60 130
[2,] 40  70
chisq.test(DTA)

Pearson's Chi-squared test with
Yates' continuity correction

data:  DTA
X-squared = 0.51854, df = 1, p-value = 0.4715


Without continuity correction:

chisq.test(DTA, cor=F)

Pearson's Chi-squared test

data:  DTA
X-squared = 0.7177, df = 1, p-value = 0.3969


Test of two proportions (two-sided alternative), with continuity correction:

prop.test(c(60,130), c(100,200))

2-sample test for equality of proportions
with continuity correction

data:  c(60, 130) out of c(100, 200)
X-squared = 0.51854, df = 1, p-value = 0.4715
alternative hypothesis: two.sided
95 percent confidence interval:
-0.17407255  0.07407255
sample estimates:
prop 1 prop 2
0.60   0.65


And without:

prop.test(c(60,130), c(100,200), cor=F)

2-sample test for equality of proportions
without continuity correction

data:  c(60, 130) out of c(100, 200)
X-squared = 0.7177, df = 1, p-value = 0.3969
alternative hypothesis: two.sided
95 percent confidence interval:
-0.16657255  0.06657255
sample estimates:
prop 1 prop 2
0.60   0.65


Output from Minitab,

Chi-squared test:

Chi-Square Test for Association

Y       N  All

A          60     130  190
63.33  126.67
0.1754  0.0877

B          40      70  110
36.67   73.33
0.3030  0.1515

All       100     200  300

Cell Contents:      Count
Expected count
Contribution to Chi-square

Pearson Chi-Square = 0.718, DF = 1,
P-Value = 0.397
Likelihood Ratio Chi-Square = 0.714, DF = 1,
P-Value = 0.398


Test of two proportions (two-sided alternative).

MTB > PTwo 100 60 200 130;
SUBC>   Confidence 95.0;
SUBC>   Test 0.0;
SUBC>   Alternative 0;
SUBC>   Pooled.
# Pooled estimate of common probability

Test and CI for Two Proportions

Sample    X    N  Sample p
1        60  100  0.600000
2       130  200  0.650000

Difference = p (1) - p (2)
Estimate for difference:  -0.05
95% CI for difference:  (-0.166573, 0.0665726)
Test for difference = 0 (vs ≠ 0):
Z = -0.85  P-Value = 0.397

Fisher’s exact test (two-sided alternative):
P-Value = 0.446


.

MTB > PTwo 100 60 200 130;
SUBC>   Confidence 95.0;
SUBC>   Test 0.0;
SUBC>   Alternative 0.
# Standard error uses 2 separate estimates

Test and CI for Two Proportions

Sample    X    N  Sample p
1        60  100  0.600000
2       130  200  0.650000

Difference = p (1) - p (2)
Estimate for difference:  -0.05
95% CI for difference:  (-0.166573, 0.0665726)
Test for difference = 0 (vs ≠ 0):
Z = -0.84  P-Value = 0.401

Fisher’s exact test: P-Value = 0.446


Notes: [1] Because of the squaring involved in computing the chi-squared test statistic, all chi-squared tests are inherently two-sided (even though one rejects in the right tail of the chi-squared distribution). [2] The likelihood ratio test uses an asymptotic chi-squared distribution. Pearson's chi-squared test uses a different approximation to a chi-squared distribution. [3] Fisher's exact test uses an exact hypergeometric distribution for one-sided tests; most computer implementations double the P-value for a two-sided test (as shown here). [4] Be careful to understand how each program expects data to be entered. For 'tests of two proportions', some programs expect you to enter counts of 'success' and 'failures'; other expect 'successes' and 'total trials'.

• This was a brilliant answer, and thanks for taking the time to generate a sample data set and to run the different tests on it! – Freya Aug 7 '19 at 16:29