# Is there any hypothesis test for two binomial distribution without normal approximation?

Let's say we are running an A/B testing and each data point has a binary response. We would like to test whether the ratio of true are different between A and B. (e.g. ask a yes/no question to both group A and group B and would like to test if there is difference in the ratio of "Yes" between the two groups)

I understand I can apply z-test if we can approximate the distribution of the number of true data (modeled as binomial distribution) as normal distribution, but there are cases that we can not approximate the binomial distribution as normal distribution.

So my question is, is there any statistical test available for given two binomial distributions $$A \sim \mathrm{Bin}(n, p_a)$$ and $$B \sim \mathrm{Bin}(m, p_b)$$ where $$n$$ and $$m$$ are the sample size of A and B to test if $$p_a$$ and $$p_b$$ are different without approximation to normal/Poisson distribution?

• Fishers test or Chi-square test? Jun 8 '20 at 12:58
• A Chi-square test still makes a distributional assumption. It assumes the sample size is large enough that the test statistic is Chi-square distributed. It's the same problem as assuming a normal approximation. I agree that Fishers test is appropriate though.
– Eli
Jun 8 '20 at 13:47

Not super related, but if you're thinking of difference of binomials, it's nice to convince yourself that if $$p_1 \neq p_2$$, then the difference is not itself a binomial! I think that's kinda fun to think about.
So my question is, is there any statistical test available for given two binomial distributions $$A \sim \mathrm{Bin}(n, p_a)$$ and $$B \sim \mathrm{Bin}(m, p_b)$$ where $$n$$ and $$m$$ are the sample size of A and B to test if $$p_a$$ and $$p_b$$ are different without approximation to normal/Poisson distribution?
The alternative is using Barnard's test, which considers a range of null hypotheses $$A \sim B \sim \mathrm{Bin}(n, p)$$ where $$p$$ is an unknown (nuisance) parameter, and the test is done by selecting the worst case (highest p-value) out of all possible $$p$$.