Post hoc Chi square non significant

Question

I am interested to see if there is any relationship between eye color and whether someone is a vegan or not. The data is as follows:

      Vegan Non vegan
Blue     29        23
Green    15        32
Brown     3         9

I ran a Chi-squared test of association and the result is significant.

    Pearson's Chi-squared test

data:  vegan
X-squared = 7.4115, df = 2, p-value = 0.02458

However, when I ran the post-hoc of this, it turns out the p-values are not significant.

Adjusted p-values used the bonferroni method.

       comparison  raw.p  adj.p
1  Blue vs. Green 0.0255 0.0766
2  Blue vs. Brown 0.1069 0.3206
3 Green vs. Brown 0.7388 1.0000

What does this mean? If I look at the counts and expected counts of each group, I can see that people who are vegan tend to have blue eyes, and non-vegan tend to have green eyes.

R code you can try:

library(fifer)
vegan <-
    matrix(c(29,15,3,23,32,9),
           nrow = 3,
           dimnames =
           list(c("Blue", "Green", "Brown"),
                c("Vegan", "Non vegan")))
chisq.test(vegan)
chisq.post.hoc(vegan, control = "bonferroni")

Answer 1

It's due to the Bonferroni correction, which controls family-wise error rates. But, like all efforts to control that error rate, the cost is reduced statistical power. Applying the correction reduces the chance of a false positive and increases the chance of getting a false negative, which is consistent with what you have observed.

For interpretation, overall your data suggests that there is some statistically significant difference in number of vegans by eye color, but the pairwise comparisons of each category (using the Bonferroni correction) can't tell you which specific categories differ from each other.

Answer 2

The reason for this is because more complex combination comparisons (contrasts) should be considered. When the omnibus chi-square test is significant, at least one 'contrast' among population proportions is significantly different but this does not have to be a simple pairwise difference. In other words, when a null hypothesis of homogeneity of proportions is rejected there may be no pair-wise different significantly contributing to this rejection. The omnibus rejection may be due to complex contrasts in population proportions (or perhaps the overall sample size is too large!) See for example https://doi.org/10.1177%2F0013164493053004008