# Comparing two association matrices (contingency tables)

I am looking for a significance test to compare between two association matrices (contingency tables) - preferably to test whether the odds ratio is the same. I have looked at some questions here but they don't provide the answer I am looking for.

For example, I would like to compare the following tables:

(Apologies the tables are presented as an image. I couldn't get MathJax to render them as tables)

What would be an appropriate test to use?

• What is your hypothesis? That the odds ratio is the same in both tables, or ...? Jul 25, 2023 at 1:13
• @kjetilbhalvorsen - yes, sorry, I forgot to add that in my haste - testing that the odds ratio is the same in both tables. I will edit my question. Jul 25, 2023 at 1:56

Firstly, notating the contingency table as:

$$\begin{array}{c|c|c|} & \text{True} & \text{False} \\ \hline \text{Before} & a & b\\ \hline \text{After} & c & d \\ \hline \end{array}$$

the Odds Ratio $$(OR)$$ is:

$$OR=\frac{ad}{bc}$$

Taking your data, for Test 1, $$OR_1=7.813$$ and for Test 2, $$OR_2=0.624$$.

The standard error $$(SE)$$ for the Odds Ratio can be approximated by:

$$SE=\sqrt{\frac{1}{a}+\frac{1}{b}+\frac{1}{c}+\frac{1}{d}}$$

For Test 1, $$SE_1=0.515$$ and for Test 2 $$SE_2=0.480$$

The confidence interval of the $$OR$$ is taken as $$e^{\ln(⁡OR)\, \pm\, z\, SE}$$.

If we set $$z=1.96$$ (i.e. 95% confidence level), we have the following confidence intervals:

$$\begin{array}{c|c|c|} & \text{Lower} & \text{Upper} \\ \hline \text{Test 1 CI} & 2.848 & 21.428\\ \hline \text{Test 2 CI} & 0.243 & 1.599 \\ \hline \end{array}$$

As the confidence interval for the tests do not overlap, we can conclude there is a significant difference in the odds ratio between the tests at the 95% confidence level.

One question remains: what is the p-value? (assuming that is what you are after). To find this, we need to determine the point where the confidence interval from one test intersects the confidence interval from the other test. This occurs when:

$$e^{\ln(⁡OR_i)\, -\, z\, SE_i}=e^{\ln(⁡OR_j)\, +\, z\, SE_j}$$ For convention, let $$i$$ be the test with the greatest $$OR$$ and $$j$$ be the test with the lowest $$OR$$. This can be solved for the absolute value of $$z$$ as follows:

$$z=\left| \frac{\ln OR_i-\ln OR_j}{SE_i+SE_j} \right |$$

Inserting the relevant data $$z=\left | \frac{\ln(7.813)-\ln⁡(0.624)}{0.515+0.480} \right |\,=\,2.541$$ Taking the standard normal distribution, this equates to a p-value (2-sided) of 0.011.

• Thanks for your answer. I can follow the approach you have taken. Jul 29, 2023 at 13:18

You can do this via logistic regression. First we must put your data in a convenient form, in R:

yourdata <- "Test  Period Outcome w
test1 before TRUE    25
test1 before FALSE   14
test1 after  TRUE    8
test1 after  FALSE   35
test2 before TRUE    13
test2 before FALSE   18
test2 after  TRUE    22
test2 after  FALSE   19 "

stringsAsFactors=TRUE)

Then building a logistic regression model, with an interaction term that represents the difference in odds ratio between the two tables:

mod0 <- glm(Outcome ~ Test * Period, weight=w, data=dat, family=binomial)
> summary(mod0)

Call:
glm(formula = Outcome ~ Test * Period, family = binomial, data = dat,
weights = w)

Deviance Residuals:
1       2       3       4       5       6       7       8
4.715  -5.356   5.187  -3.796   4.753  -4.424   5.234  -5.406

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)             -1.4759     0.3919  -3.766 0.000166 ***
Testtest2                1.6225     0.5017   3.234 0.001219 **
Periodbefore             2.0557     0.5148   3.993 6.51e-05 ***
Testtest2:Periodbefore  -2.5278     0.7040  -3.591 0.000330 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 211.38  on 7  degrees of freedom
Residual deviance: 191.02  on 4  degrees of freedom
AIC: 199.02

Number of Fisher Scoring iterations: 5

EDIT There might be other approaches. You have two contingency tables, if you lay one over the other (using Test as the layer variable) you get a three-way contingency table, that could be a search term. There are specific hypothesis tests for that situation, but probably an approach like above using generalized linear models is preferred today. Some links you could look at:

• +1 This is what I did for contingency table analysis a few years ago, and I am glad to see someone reputable saying to do the same. I also did a chunk test of the test2 variable and the interaction as a full test of contingency table equivalence. I didn’t care what the odds ratios in the tables were, as long as they differed (more or less).
– Dave
Jul 26, 2023 at 22:34
• Appreciate taking the time to answer. I am still learning advanced stats and R so some of this is new for me. Jul 27, 2023 at 21:27
• I added some info ... Jul 27, 2023 at 22:29