Post hoc Chi square non significant

I am interested to see if there is any relationship between eye color and whether someone is a vegan or not. The data is as follows:

      Vegan Non vegan
Blue     29        23
Green    15        32
Brown     3         9


I ran a Chi-squared test of association and the result is significant.

    Pearson's Chi-squared test

data:  vegan
X-squared = 7.4115, df = 2, p-value = 0.02458


However, when I ran the post-hoc of this, it turns out the p-values are not significant.

Adjusted p-values used the bonferroni method.

1  Blue vs. Green 0.0255 0.0766
2  Blue vs. Brown 0.1069 0.3206
3 Green vs. Brown 0.7388 1.0000


What does this mean? If I look at the counts and expected counts of each group, I can see that people who are vegan tend to have blue eyes, and non-vegan tend to have green eyes.

R code you can try:

library(fifer)
vegan <-
matrix(c(29,15,3,23,32,9),
nrow = 3,
dimnames =
list(c("Blue", "Green", "Brown"),
c("Vegan", "Non vegan")))
chisq.test(vegan)
chisq.post.hoc(vegan, control = "bonferroni")


It's due to the Bonferroni correction, which controls family-wise error rates. But, like all efforts to control that error rate, the cost is reduced statistical power. Applying the correction reduces the chance of a false positive and increases the chance of getting a false negative, which is consistent with what you have observed.

For interpretation, overall your data suggests that there is some statistically significant difference in number of vegans by eye color, but the pairwise comparisons of each category (using the Bonferroni correction) can't tell you which specific categories differ from each other.

The reason for this is because more complex combination comparisons (contrasts) should be considered. When the omnibus chi-square test is significant, at least one 'contrast' among population proportions is significantly different but this does not have to be a simple pairwise difference. In other words, when a null hypothesis of homogeneity of proportions is rejected there may be no pair-wise different significantly contributing to this rejection. The omnibus rejection may be due to complex contrasts in population proportions (or perhaps the overall sample size is too large!) See for example https://doi.org/10.1177%2F0013164493053004008

I used the function RVAideMemoire::chisq.multcomp and found that actually

multveg <- RVAideMemoire::chisq.multcomp(vegan, p.method = "bonferroni")


    Pairwise comparisons using chi-squared tests

data:  vegan

3                    9                   15                  23                  29
9  1.00000000000000000  -                   -                   -                   -
15 0.07016602471570901  1.00000000000000000 -                   -                   -
23 0.00131549135633853  0.19992493171226339 1.00000000000000000 -                   -
29 6.45416919551268e-05 0.01765298865937108 0.52212718217800613 1.00000000000000000 -
32 1.42375085982061e-05 0.00492231343127806 0.19723675996345547 1.00000000000000000 1.00000000000000000



From those p-values we can identify which are lower than 0.05:

   > which(multveg\$p.value < 0.05, arr.ind = TRUE)
row col
23   3   1
29   4   1
32   5   1
29   4   2
32   5   2


The matrix names are taken from cell values, so let's remember the "cells names":

      Vegan Non vegan
Blue     29        23
Green    15        32
Brown     3         9


That means, this combinations are different than expected:

Group 23: row:3 col:1 group: 23-3 Blue|NoVeg vs Brown|vegan

• Group 29: row:4 col:1 group: 29-3 Blue|Veg vs Brown|vegan
• Group 32: row:5 col:1 group: 32-3 Green|NoVeg vs Brown|vegan
• Group 29: row:4 col:2 group: 29-9 Blue|Veg vs Brown|NoVegan
• Group 32: row:5 col:2 group: 32-9 Green|NoVeg vs Brown|NoVegan