# How to carry out multiple post-hoc chi-square tests on a 2 X 3 table?

My data set is comprised of either total mortality or survival of an organism at three site types, inshore, midchannel and offshore. The numbers in the table below represent the number of sites.

              100% Mortality            100% Survival
Inshore             30                       31
Midchannel          10                       20
Offshore             1                       10


I would like to know if the # of sites where 100% mortality occurred is significant based on site type. If I run a 2 x 3 chi-square, I get a significant result. Is there a post-hoc pairwise comparison that I can run or should I actually be using a logistical ANOVA or regression with binomial distribution? Thanks!

A contingency table should contain all the mutually exclusive categories on both axes. Inshore/Midchannel/Offshore look fine, however unless "less than 100% mortality" means "100% survival" in this biological setting you may need to construct tables that account for all the cases observed or explain why you restrict your analysis to the extreme ends of the sample.

As 100% survival means 0% mortality, you could have a table with columns 100%=mortality / 100%>mortality>0% / mortality=0%. In this case you wouldn't any more compare percentages, but compare ordinal mortality measures across three site type categories. (What about using the original percentage values instead of categories?) A version of Kruskal-Wallis test may be appropriate here that takes ties appropriately into consideration (maybe a permutation test).

There are established post hoc tests for the Kruskal-Wallis test: 1, 2, 3. (A resampling approach may help tackling with ties.)

Logistic regression and binomial regression may be even better as they not only give you p values, but also useful estimates and confidence intervals of the effect sizes. However to set up those models more details would be needed concerning the 100%>mortality>0% sites.

I am going to assume that "100% survival" means that your sites only contained a single organism. so 30 means 30 organisms died, and 31 means 31 organisms didn't. Based on this the chi-square should be fine, but it will only tell which hypothesis are not supported by the data - it won't tell you if two reasonable hypothesis are better or not. I present a probability analysis which does extract this information - it agrees with the chi-square test, but it gives you more information than the chi-square test, and a better way to present the results.

The model is a bernouli model for the indicator of "death", $Y_{ij}\sim Bin(1,\theta_{ij})$ ($i$ denotes the cell of the $2\times 3$ table, and $j$ denotes the individual unit within the cell).

There are two global assumption underlying the chi-square test:

1. within a given cell of the table, the $\theta_{ij}$ are all equal, that is $\theta_{ij}=\theta_{ik}=\theta_{i}$
2. the $Y_{ij}$ are statistically independent, given $\theta_{i}$. This means that the probability parameters tell you everything about $Y_{ij}$ - all other information is irrelevant if you know $\theta_{i}$

Denote $X_{i}$ as the sum of $Y_{ij}$, (so $X_{1}=30,X_{2}=10,X_{3}=1$) and let $N_{i}$ be the group size (so $N_{1}=61,N_{2}=30,N_{3}=11$). Now we have a hypothesis to test:

$$H_{A}:\theta_{1}=\theta_{2},\theta_{1}=\theta_{3},\theta_{2}=\theta_{3}$$

But what are the alternatives? I would say the other possible combinations of equal or not equal.

$$H_{B1}:\theta_{1}\neq\theta_{2},\theta_{1}\neq\theta_{3},\theta_{2}=\theta_{3}$$ $$H_{B2}:\theta_{1}\neq\theta_{2},\theta_{1}=\theta_{3},\theta_{2}\neq\theta_{3}$$ $$H_{B3}:\theta_{1}=\theta_{2},\theta_{1}\neq\theta_{3},\theta_{2}\neq\theta_{3}$$ $$H_{C}:\theta_{1}\neq\theta_{2},\theta_{1}\neq\theta_{3},\theta_{2}\neq\theta_{3}$$

One of these hypothesis has to be true, given the "global" assumptions above. But note that none of these specify specific values for the rates - so they must be integrated out. Now given that $H_{A}$ is true, we only have one parameter (because all are equal), and the uniform prior is a conservative choice, denote this and the global assumptions by $I_{0}$. so we have:

$$P(X_{1},X_{2},X_{3}|N_{1},N_{2},N_{3},H_{A},I_{0})=\int_{0}^{1}P(X_{1},X_{2},X_{3},\theta|N_{1},N_{2},N_{3},H_{A},I_{0})d\theta$$ $$={N_{1} \choose X_{1}}{N_{2} \choose X_{2}}{N_{3} \choose X_{3}}\int_{0}^{1}\theta^{X_{1}+X_{2}+X_{3}}(1-\theta)^{N_{1}+N_{2}+N_{3}-X_{1}-X_{2}-X_{3}}d\theta$$ $$=\frac{{N_{1} \choose X_{1}}{N_{2} \choose X_{2}}{N_{3} \choose X_{3}}}{(N_{1}+N_{2}+N_{3}+1){N_{1}+N_{2}+N_{3} \choose X_{1}+X_{2}+X_{3}}}$$

Which is a hypergeometric distribution divided by a constant. Similarly for $H_{B1}$ we will have: $$P(X_{1},X_{2},X_{3}|N_{1},N_{2},N_{3},H_{B1},I_{0})=\int_{0}^{1}P(X_{1},X_{2},X_{3},\theta_{1}\theta_{2}|N_{1},N_{2},N_{3},H_{B1},I_{0})d\theta_{1}d\theta_{2}$$ $$=\frac{{N_{2} \choose X_{2}}{N_{3} \choose X_{3}}}{(N_{1}+1)(N_{2}+N_{3}+1){N_{2}+N_{3} \choose X_{2}+X_{3}}}$$

You can see the pattern for the others. We can calculate the odds for say $H_{A}\;vs\;H_{B1}$ by simply dividing the above two expressions. The answer is about $4$, which means the data support $H_{A}$ over $H_{B1}$ by about a factor of $4$ - fairly weak evidence in favour of equal rates. The other probabilities are given below.

$$\begin{array}{c|c} Hypothesis & probability \\ \hline (H_{A}|D) & 0.018982265 \\ (H_{B1}|D) & 0.004790669 \\ (H_{B2}|D) & 0.051620022 \\ (H_{B3}|D) & 0.484155874 \\ (H_{C}|D) & 0.440451171 \\ \end{array}$$

This is showing strong evidence against equal rates, but not in strong evidence favour of a defintie alternative. It seems like there is strong evidence that the "offshore" rate is different to the other two rates, but inconclusive evidence as to whether "inshore" and "mid-channel" rates differ. This is what the chi-square test won't tell you - it only tells you that hypothesis $A$ is "crap" but not what alternative to put in its place

Here is the code to do the chi square tests as well as generate a variety of test statistics. However, statistical tests of association of the table margins are useless here; the answer is obvious. No one does a statistical test to see if summer is hotter than winter.

Chompy<-matrix(c(30,10,1,31,20,10), 3, 2)
Chompy
chisq.test(Chompy)
chisq.test(Chompy, simulate.p.value = TRUE, B = 10000)
chompy2<-data.frame(matrix(c(30,10,1,31,20,10,1,2,1,2,1,2,1,2,3,1,2,3), 6,3))
chompy2
chompy2$X2<-factor(chompy2$X2)
chompy2$X3<-factor(chompy2$X3)
summary(fit1<-glm(X1~X2+X3, data=chompy2, family=poisson))
summary(fit2<-glm(X1~X2*X3, data=chompy2, family=poisson)) #oversaturated
summary(fit3<-glm(X1~1, data=chompy2, family=poisson)) #null
anova(fit3,fit1)
library(lmtest)
waldtest(fit1)
waldtest(fit2) #oversaturated
kruskal.test(X1~X2+X3, data=chompy2)
kruskal.test(X1~X2*X3, data=chompy2)

• It would be interesting for the reader (and the OP) if you could provide details about the different R syntax (and the underlying tests) you gave, and especially how a Kruskal-Wallis test does compare to a log-linear model. – chl Apr 18 '11 at 19:38
• You can see this by copying and pasting the code into the R console. – Patrick McCann Apr 29 '11 at 23:15
• Sure. Responses come from themselves by running the code, of course. – chl May 1 '11 at 9:41

I believe you could use the "simultaneous confidence intervals" for doing multiple comparisons. The reference is Agresti et al. 2008 Simultaneous confidence intervals for comparing binomial parameters. Biometrics 64 1270-1275.

You could find the corresponding R code in http://www.stat.ufl.edu/~aa/cda/software.html