# Comparison of statistical tests exploring co-dependence of two binary variables

Suppose we observe data $$(X_i,Y_i)_{i=1,...,n}$$ on two binary variables: $$X\in\{0,1\}$$ and $$Y\in\{0,1\}$$. We would like to test if $$X$$ and $$Y$$ are co-dependent (related). Standard suggestions in mainstream textbooks are the following:

• chi-square test for independence of $$X$$ and $$Y$$,
• Z-test for comparing proportions of $$[Y = 1]$$ between two groups: $$[X = 0]$$ and $$[X = 1]$$,
• Z-test for comparing proportions of $$[X = 1]$$ between two groups: $$[Y = 0]$$ and $$[Y = 1]$$.

In addition to that, we can run logistic regressions of $$Y$$ on $$X$$ and $$X$$ on $$Y$$. We can check statistical significance of the slope coefficients. There are at least $$3$$ standard tests for that: likelihood ratio, Wald and deviance. Since we consider two regressions, there are $$3 * 2 = 6$$ tests added, making the total number $$9$$. But wait, we can run probit models too. Et cetera, et cetera, ...

Is there one or more references which systematically and rigorously answer(s) the following questions:

• Which tests are algebraically equivalent and when?
• Which tests are most powerful, why and when?
• What is the power function for each test and each sample size?
• In practical terms, which tests deliver the same verdict almost always?
• also I remember logistic regression and test being equivalent to z test on log-odds, which should be the same as chi square test. but I'm not sure of what kind on test on the logistic model, most probably wald one – carlo Nov 20 '19 at 13:56
• I don't know such literature, but I could offer to do my own analysis with a simulation: It will reveal if some of these methods (always) give identical outputs. Afterwards, I should have the mathematical intuition to argue why the ones, which are different, behave like they do. – KaPy3141 Nov 26 '19 at 11:32
• Thank you for the offer. Simulation is always a possibility. However, accurate simulation on a big grid of parameter values and sample sizes would be a sizable task. Even then it would not necessarily provide intuition comparable to formulas... A good part of the problem must have been solved analytically somewhere. For the rest, there must have been extensive simulation studies over the previous decades. – stans - Reinstate Monica Nov 26 '19 at 16:11

To me, this seems like a case where you should use Fisher's exact test.

I.e. you would make a table of your outcomes,

╔═════╦═════════════╦═════════════╗
║     ║     X=0     ║     X=1     ║
╠═════╬═════════════╬═════════════╣
║ Y=0 ║ |[X=0^Y=0]| ║ |[X=1^Y=0]| ║
║ Y=1 ║ |[X=0^Y=1]| ║ |[X=1^Y=1]| ║
╚═════╩═════════════╩═════════════╝


where e.g. |[X=0^Y=0]| are the number of data points with $$X_i$$=0 and $$Y_i$$=0.

As explained in the wikipedia entry, the values in the table will follow a hypergeometric distribution.

• Could you elaborate please? – Michael R. Chernick Dec 18 '19 at 17:59