Does a 2*2 chi-square test give the same results as a binomial test?

Question

This is a conceptual question. Suppose we have an experimental design with two grouping variables (e.g., the subjects are either male or female) and the variable we are measuring is a binary variable, like whether they are smokers or not - and we are interested in seeing whether the prevalence of smoking depends on gender.

I feel like both a binomial test and a 2*2 chi-square test would work in this case. For example, we could look at the proportion of yes responses in women and in men, and then compare whether one proportion was significantly different from the other (i.e., the binomial test). Alternatively, we could have a 2*2 contingency table with male/female and smoker/non-smoker and we could do a chi-square test.

My question is, would these two tests give the same results?

What would be the principled way of deciding which test to use in this situation?

Answer 1

The basic concept is that they should be the same. However, different statistical programs use slight variations that give different answers. Several variations are illustrated here. I have chosen data for which there is no significant effect. (In my experience, P-values vary less in the case of significant effects.)

My fake data are as follows:

Gender\Smoke  Y   N   Tot
F            60  40   100 
M           130  70   200

Output from R: Chi-squared test with Yates continuity correction.

a = c(60,40); b = c(130,70)
DTA = cbind(a,b); DTA
      a   b
[1,] 60 130
[2,] 40  70
chisq.test(DTA)

        Pearson's Chi-squared test with 
         Yates' continuity correction

data:  DTA
X-squared = 0.51854, df = 1, p-value = 0.4715

Without continuity correction:

chisq.test(DTA, cor=F)

        Pearson's Chi-squared test

data:  DTA
X-squared = 0.7177, df = 1, p-value = 0.3969

Test of two proportions (two-sided alternative), with continuity correction:

prop.test(c(60,130), c(100,200))

        2-sample test for equality of proportions 
        with continuity correction

data:  c(60, 130) out of c(100, 200)
X-squared = 0.51854, df = 1, p-value = 0.4715
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.17407255  0.07407255
sample estimates:
prop 1 prop 2 
  0.60   0.65 

And without:

prop.test(c(60,130), c(100,200), cor=F)

        2-sample test for equality of proportions 
        without continuity correction

data:  c(60, 130) out of c(100, 200)
X-squared = 0.7177, df = 1, p-value = 0.3969
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.16657255  0.06657255
sample estimates:
prop 1 prop 2 
  0.60   0.65 

Output from Minitab,

Chi-squared test:

Chi-Square Test for Association 

            Y       N  All

A          60     130  190
        63.33  126.67
       0.1754  0.0877

B          40      70  110
        36.67   73.33
       0.3030  0.1515

All       100     200  300

Cell Contents:      Count
                    Expected count
                    Contribution to Chi-square


Pearson Chi-Square = 0.718, DF = 1, 
    P-Value = 0.397
Likelihood Ratio Chi-Square = 0.714, DF = 1, 
    P-Value = 0.398

Test of two proportions (two-sided alternative).

MTB > PTwo 100 60 200 130;
SUBC>   Confidence 95.0;
SUBC>   Test 0.0;
SUBC>   Alternative 0;
SUBC>   Pooled.          
        # Pooled estimate of common probability

Test and CI for Two Proportions 

Sample    X    N  Sample p
1        60  100  0.600000
2       130  200  0.650000


Difference = p (1) - p (2)
Estimate for difference:  -0.05
95% CI for difference:  (-0.166573, 0.0665726)
Test for difference = 0 (vs ≠ 0):  
   Z = -0.85  P-Value = 0.397

Fisher’s exact test (two-sided alternative): 
   P-Value = 0.446

.

MTB > PTwo 100 60 200 130;
SUBC>   Confidence 95.0;
SUBC>   Test 0.0;
SUBC>   Alternative 0.   
      # Standard error uses 2 separate estimates

Test and CI for Two Proportions 

Sample    X    N  Sample p
1        60  100  0.600000
2       130  200  0.650000


 Difference = p (1) - p (2)
 Estimate for difference:  -0.05
 95% CI for difference:  (-0.166573, 0.0665726)
 Test for difference = 0 (vs ≠ 0):  
     Z = -0.84  P-Value = 0.401

 Fisher’s exact test: P-Value = 0.446

Notes: [1] Because of the squaring involved in computing the chi-squared test statistic, all chi-squared tests are inherently two-sided (even though one rejects in the right tail of the chi-squared distribution). [2] The likelihood ratio test uses an asymptotic chi-squared distribution. Pearson's chi-squared test uses a different approximation to a chi-squared distribution. [3] Fisher's exact test uses an exact hypergeometric distribution for one-sided tests; most computer implementations double the P-value for a two-sided test (as shown here). [4] Be careful to understand how each program expects data to be entered. For 'tests of two proportions', some programs expect you to enter counts of 'success' and 'failures'; other expect 'successes' and 'total trials'.