# Pooled variance for post hoc tests for contingency table $\chi^{2}$ tests

The 2-by-2 contingency table test provides an alternate hypothesis testing approach to the $z$ test for proportion difference. Conveniently enough, a 2-by-k contingency table test provides an omnibus test for proportion difference, akin to the one-way ANOVA's test for mean difference:

One-way ANOVA $H_{0}: \mu_{1} = \mu_{2} = \cdots = \mu_{k}$

2-by-$k$ contingency table test $H_{0}: p_{1} = p_{2} = \cdots = p_{k}$

If we reject the null hypothesis in the 2-by-$k$ contingency table test, we could proceed to conduct post hoc pairwise comparisons using the z test for proportion difference between groups $i$ and $j$, where the test statistic is given by:

$$z = \frac{\hat{p}_{i}-\hat{p}_{j}}{\sqrt{\hat{p}\left(1-\hat{p}\right)\left[\frac{1}{n_{i}}+\frac{1}{n_{j}}\right]}}$$

And where (I think) $\hat{p}$ creates the pooled estimate assuming the contingency table's null hypothesis is true (i.e. $\hat{p}$ is the total number of events divided by the total sample size across all $k$ groups).

Question 1: How to incorporate this pooled estimate into post hoc pairwise 2-by-2 contingency table tests (i.e. $\chi^{2}$ tests)? (bonus points if you can speak to continuity corrections)

Contingency table tests are also useful for posing questions about evidence of association between two categorical variables where both have more than 2 categories. For an $l$-by-$k$ contingency table test, where $l>2$ and $k>2$, if we reject the null hypothesis and wish to proceed to conduct post hoc subgroup tests:

Question 2a: How do we incorporate the pooled variance under the $l$-by-$k$ contingency table test's null hypothesis for post hoc 2-by-2 table tests (i.e. $\chi^{2}$ tests)? (bonus for continuity corrections)

Question 2b: How do we incorporate the pooled variance under the $l$-by-$k$ contingency table test's null hypothesis for post hoc $m$-by-$n$ contingency table tests (i.e. $\chi^{2}$ tests), either $2<m\le l$ OR $2<n\le k$, or $2<m\le l$ AND $2<n\le k$, and these tests ARE disjoint (i.e. they do not overlap on the $l$-by-$k$ contingency table).

Question 2c: How do we incorporate the pooled variance under the $l$-by-$k$ contingency table test's null hypothesis for post hoc $m$-by-$n$ contingency table tests (i.e. $\chi^{2}$ tests), either $2<m\le l$ OR $2<n\le k$, or $2<m\le l$ AND $2<n\le k$, and these tests ARE NOT disjoint (i.e. they do overlap on the $l$-by-$k$ contingency table).

• To my (non-expert) gut this looks like something that's difficult to do properly in frequentist setting. Would sketching a Bayesian solution for this class of problems be an acceptable answer or are you confined to the frequentist approach? – Martin Modrák Mar 9 at 19:52
• @MartinModrák Frequentist. Thank you for asking. If there's no feasible frequentist answer, an explanation of why would be an acceptable answer for me. – Alexis Mar 9 at 20:40
• I don't think it is infeasible or impossible, just that in my experience with frequentist settings, small variations in the question you ask often require deriving novel formulas which might or might not be found in some half-forgotten paper from 30 years ago :-) (you know, just the usual Bayesian propaganda) – Martin Modrák Mar 10 at 12:21
• I am not sure what the problem is with "How do we incorporate the pooled variance...". Does it not boil down to performing multiple ordinary z-tests for pairwise comparisons? (where I think $\hat{p}$ refers to the pooled probability for the i-th and j-th cell, since that is the hypothesis tested with that formula: $p_i = p_j$) – Martijn Weterings Mar 13 at 11:48
• What would be the hypothesis in the 'post hoc m-by-n contingency table tests' and why do you look for some form of z-test when a chi-squared test seems more suitable? – Martijn Weterings Mar 13 at 12:01