# May I replace Fisher's exact test or Chi-squared test with logistic regression and vice versa?

Let's assume we have two groups of patients, a control group and a treatment group. They were asked a question and the answer can be yes or no. Since I come from biological science where I used mainly regression methods I would use logistic regression. A friend of mine from the sociological sciences would analyze this data using Chi-square test or Fisher's exact test. Now, I'm wondering if there are reason to use the one or the other method.

To clarify my question, I created a dataset which I analyzed with several methods. The R-code is also mentioned below. Here are the results (odds ratio and p-values) of the analyses:

                                odds ratio      p-value     comment
manually (formula)               0.474        0.324     see details below
chi-square                       -            0.524     warnings: approx. incorrect
chi-square (simulated)           -            0.460
fisher's exact test              0.477        0.465
logistic regression              0.474        0.324     exact what I get manually
Bayes logistic regression        0.526        0.356     shrinking effect of Bayes


As you can see the results are mostly similar. But imagine the situation if I want to publish the results performed with logistic regression and a reviewer asks me why I don't use the "classical" Chi-square test or Fisher's exact test and vice versa?

Here is how I created the data set and the analysis:

library(arm)
set.seed(12345)

# size of groups
n <- 50  # control
m <- 60  # treatment

# Create data
df0 <- data.frame(group = c(rep("ctrl",n), rep("treat",m))
, out = c(rbinom(n=n, 1, 0.1), rbinom(n=m,1,0.03))
)

# Tabulate
with(df0,table(group, out, useNA='ifany'))

out
group    0  1
ctrl  45  5
treat 57  3

# Convert to matrix for using Fisher's test or Chi-square test
mx0 <- with(df0,table(group, out, useNA='ifany'))

# Are there expected values lower 5?
round(chisq.test(mx0)$exp,1) out group 0 1 ctrl 46.4 3.6 treat 55.6 4.4 # Chi-square test chisq.test(mx0)$$p.value # 0.5242444 chisq.test(mx0, simulate.p.value=TRUE)$$p.value # 0.4602699 # Fisher's exact test fisher.test(mx0)$$p.value # 0.4646205 fisher.test(mx0)$$estimate # 0.4769107 fisher.test(mx0)$conf.int[1:2]                  # 0.0703001 2.6013230

# manually (OR and CI):
(or <- prod(diag(mx0)) / prod(diag(apply(mx0,2,rev))) ) # 0.4736842
# log of standard error: root of (1/a+1/b+1/c+1/d):
se.ln <- sqrt(sum(1/mx0))
(cil <- exp(log(or) - qnorm(0.975)*se.ln))              # 0.1074239
(ciu <- exp(log(or) + qnorm(0.975)*se.ln))              # 2.088704
(t <- abs(log(or)/se.ln))
2*pnorm(t, lower.tail=FALSE)                           # 0.323628

# Logistic regression
(fit1 <- summary(glm(out ~ group, data=df0, family="binomial"))$coef) # Estimate Std. Error z value Pr(>|z|) # (Intercept) -2.1972246 0.4714045 -4.6610172 3.146504e-06 # grouptreat -0.7472144 0.7570094 -0.9870609 3.236128e-01 exp(fit1[2,"Estimate"]) # 0.4736842 (fit2 <- summary(bayesglm(out ~ group, data=df0, family="binomial"))$coef)

#               Estimate Std. Error    z value     Pr(>|z|)
# (Intercept) -2.2326782  0.4645022 -4.8066039 1.535157e-06
# grouptreat  -0.6415988  0.6947115 -0.9235471 3.557222e-01

exp(fit2[2,"Estimate"])        # 0.5264501

• what are your specific objectives of study? Also, you may state the hypotheses ?
– user10619
Commented Nov 9, 2018 at 5:07

Fisher’s “exact test” is not very accurate so I never use it.

In maximum likelihood estimation such as what is used for logistic models, there are 3 test statistics: the preferred likelihood ratio $$\chi^2$$ test, Rao efficient score tests, and the most commonly used but often problematic Wald test. In the special case where there is only one predictor in the logistic model and the predictor is categorical, the score test for that predictor in a binary or multinomial logistic model is exactly equal to the classic Pearson $$\chi^2$$ statistic. When the outcome is ordinal and is analyzed with a proportional odds ordinal logistic model, the score $$\chi^2$$ from that ordinal model is exactly the Wilcoxon/Kruskal-Wallis statistic.

We no longer need the special cases like contingency table $$\chi^2$$, Wilcoxon, and log-rank tests.

• I was about to post this...reviewer asks why you used a logistic regression instead of a chi-squared test, point out that a chi-squared test is a logistic regression (though there could be some legitimacy to a criticism of using a software-default Wald test like you get from glm in R).
– Dave
Commented Jun 21 at 11:47

Appreciate the detailed gathering of comparative contingency table tests. Just FYI, the above results actually correspond to the following table counts:

          out
group    0  1
ctrl  45  5
treat 57  3


One exception:

> chisq.test(mx0, simulate.p.value=TRUE)\$p.value  # 0.4602699
[1] 0.4487756

• You're right. No idea how this could happened. I corrected the table. The simulated p-value will change any time I run it. Here is a squence of runs: 0.4557721, 0.4687656,0.4542729, 0.4567716, 0.4647676. Commented Jun 23 at 15:14

This is an interesting question. While there are subtle differences between the approches, they will often get similar answer. But note that the standard logistic regression analyses is asymptotic, so with few observations the other methods could be preferred, or you could maybe bootstrap the logistic regression.

• They are all valid although the Fisher Exact test is usually overly conservative. You should choose one a priori and stick with that. I wouldn't switch based on a reviewer 's comments. Commented Jun 6, 2017 at 2:42