Interpretation of Odds Ratio (OR) in Fisher's exact test

Question

I've tried using R operating Fisher's exact test to test for independence of the 2 datasets below and struggling to interpret the Fisher report. Please have a look at the data and the Fisher's test below.

Dataset 1: is a 2x3 contingency table

Consider then an experiment where we have 2 categorical variables: Group (I and II) & Outcome (Worse, Same, and Improve). These results might follow the administration of a new drug. We collect data and find out the frequencies as shown in the table below.

You can reproduce Dataset 1, and run the Fisher's test using the codes:

obs <- rbind(c(33, 44, 25),
             c(11, 28, 30))
rownames(obs) <- c("Group I", "Group II")
colnames(obs) <- c("Worse", "Same", "Improve")
obs

##          Worse Same Improve
## Group I     33   44      25
## Group II    11   28      30

fisher.test(obs)

##   Fisher's Exact Test for Count Data

## data:  obs
## p-value = 0.01049
## alternative hypothesis: two.sided   

Dataset 2: is a 2x2 contingency table

Consider then an experiment where we have 2 categorical variables: Group (I and II) & Outcome (Worse and Improve). These results might follow the administration of a new drug. We collect data and find out the frequencies as shown in the table below.

vals <- matrix(c(3, 1, 3, 2),
               nrow = 2,
               dimnames = list(Group = c("Group I", "Group II"),
                   Outcome = c("Worsened", "Improved")))
##  Outcome
##  Group          Worsened Improved
##  Group I             3        3
##  Group II            1        2

fisher.test(vals)

## 
##  Fisher's Exact Test for Count Data
## 
## data:  vals
## p-value = 1
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##    0.06060903 156.52286969
## sample estimates:
## odds ratio 
##   1.852496

As you can see that I receive 2 different report templates for the two contingency tables a (2x3) and a (2x2). I have 3 questions:

1. Why the report of a 2x2 includes the Odds Ratio (OR) and the Confidence Interval, which I believe is a CI of the Odds Ratio, while the 2x3 doesn't have the OR in there?
   Does that mean testing for independence using the value of OR can be applied ONLY for 2x2 contingency tables??
2. Could you help me interpret the Fisher's test report and point out the connection between the values of p-value, Odds ratio, and the Confidence Intervals?
3. In what case will we be using "one-sided Fisher's exact test"? What is the purpose of pointing out a Fisher's test should be one-sided or two-sided?
4. Can a table that's NOT 2x2 be tested using a "one-sided Fisher's exact test"?

Answer 1

1. The odds ratio cannot be defined for more than two levels of response. If you want an odds ratio, there are two options I can think of. One, convert the response into two levels by combining two of the levels. Two, ordinal regression.

2. For your second dataset, the point estimate of the odds ratio is greater than 1 and there is slightly more evidence that the odds ratio is greater than 1 compared to the evidence that it is less than 1. There is a smaller p-value if you specify the option alternative="greater". The option chosen should depend on the goals of the research.

3. Yes, if you use the ordinal regression, the hypothesis and p-value can be one-sided. For the general test of independence, no; it is a two-sided test. fisher.test allows options, but the p-value is identical in all 3 cases:

fisher.test(obs)  
fisher.test(obs, alternative="less")  
fisher.test(obs, alternative="greater") 

You can also try

library(coin)  
cmh_test(as.table(obs))  

The p-value is slightly different from fisher.test. That is because cmh_test uses an asymptotic p-value. If you use fisher.test(obs, hybrid=T), the p-value is closer to the asymptotic p-value from cmh_test.

Answer 2

Short answer: A $2\times 2$ table has 1 degree of freedom (df), and one odds ratio (OR) correspondingly. A $2\times 3$ table has two df, and two ORs. In that case, the null hypothesis tested by Fisher's test can be formulated as both the ORs being equal to 1.

You could calculate the ORs by splitting into two $2\times 2$ tables, or simply by using a logistic regression, where the coefficents (forget the intercept) are log odds ratios. With your data, using R:

obs <- rbind(c(33, 44, 25),
             c(11, 28, 30))
rownames(obs) <- c("Group I", "Group II")
colnames(obs) <- c("Worse", "Same", "Improve")
yourdf <- as.data.frame(as.table(obs))

colnames(yourdf) <- c("Group", "Result", "Freq")
yourdf$Group <- as.factor(yourdf$Group)
yourdf$Result <- as.factor(yourdf$Result)

First we fit a binary logistic regression model (LR) using Group as response variable. That doesn't sound natural, but for a binary LR we need a two-valued response ...

mod1 <- glm(Group ~ Result,  weight=Freq, data=yourdf, family=binomial)   
 summary(mod1)

Call:
glm(formula = Group ~ Result, family = binomial, data = yourdf, 
    weights = Freq)

Deviance Residuals: 
     1       2       3       4       5       6  
-4.357   5.522  -6.583   7.272  -6.279   6.031  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)   
(Intercept)    -1.0986     0.3481  -3.156  0.00160 **
ResultSame      0.6466     0.4238   1.526  0.12710   
ResultImprove   1.2809     0.4411   2.904  0.00368 **
---

Compare the results using a multinomial LR, which looks more natural here:

library(nnet)

mod2 <- nnet::multinom( Result ~ Group, data=yourdf, weight=Freq)
summary(mod2)
Call:
nnet::multinom(formula = Result ~ Group, data = yourdf, weights = Freq)

Coefficients:
        (Intercept) GroupGroup II
Same      0.2876825     0.6466277
Improve  -0.2776313     1.2809341

Std. Errors:
        (Intercept) GroupGroup II
Same      0.2302831     0.4238558
Improve   0.2651472     0.4410731

Now compare the log ORs from the two models!

To see how closely related these two model formulations are, let us compare the likelihood ratio tests comparing to the null model (corresponding to the null hypothesis):

mod2_0 <- nnet::multinom( Result  ~ 1, data=yourdf,  weight=Freq)
anova(mod1, test="Chisq")
Analysis of Deviance Table

Model: binomial, link: logit

Response: Group

Terms added sequentially (first to last)


       Df Deviance Resid. Df Resid. Dev Pr(>Chi)  
NULL                       5     230.65           
Result  2   9.1435         3     221.50  0.01034 *

 anova(mod2_0, mod2, test="Chisq")
Likelihood ratio tests of Multinomial Models

Response: Result
  Model Resid. df Resid. Dev   Test    Df LR stat.    Pr(Chi)
1     1        10   368.7937                                 
2 Group         8   359.6502 1 vs 2     2 9.143495 0.01033987