Understanding Chi Square Post hoc test results

Question

I'm using R and comparing the proportion of Male and Female authors from various academic journals (well, I think I am). Here's a table with my raw data:

The code for it:

m<-structure(c(8278L, 2183L, 0L, 3844L, 590L, 2341L, 1659L, 422L, 
0L, 899L, 137L, 662L), .Dim = c(6L, 2L), .Dimnames = structure(list(
    c("PRx", "AXJ", "JAPA", "JSPRAS", "CPX", "PRX-TO"), c("Male", 
    "Female")), .Names = c("", "")), class = "table")

I created that table above with this code:

m<-table(data3$Gender,data3$Journal)

I run a chi-square test and get significant results:

chisq.test(data3$Journal,data3$Gender)

And when I do a chi square post hoc test in R using this code:

chisq.posthoc.test(m)

Some of the results really don't make sense to me.

For example, "PRX" has a significant value, which to me indicates that PRX is different from the other groups. But as a proportion (females are 16.69% of count for PRX), thats pretty much the same proportion as AXJ and JSPRAS (16.19% and 18% respectively) which PRX is in between. What am I not understanding correctly? It sounds like this test is showing something different than what I think it is maybe?

If I'm using the wrong test for my question, is there a better way to test it?

Answer 1

First, it's not clear why you include JAPA with $0$ counts in both columns. What useful information does that provide?

Second, if the issue is whether there are fewer female authors than male ones, with a overall ratio of $17,036:3779,$ the answer is overwhelmingly Yes with no need for a statistical test. If you must have one, here is output from prop.test in R:

prop.test(c(17036, 3779), c(20815, 20815), cor=F)

        2-sample test for equality of proportions 
        without continuity correction

data:  c(17036, 3779) out of c(20815, 20815)
X-squared = 16887, df = 1, p-value < 2.2e-16
alternative hypothesis: two.sided
95 percent confidence interval:
 0.6294907 0.6443022
sample estimates:
   prop 1    prop 2 
0.8184482 0.1815518 

Third, if the question is whether the male to female ratio differs by journal, the overall answer is Yes, on account of the tiny P-value for a test of homogeneity on TBL.

TBL = rbind(c(8278, 2183, 3844, 390, 2341),
            c(1659, 422, 899, 137, 662))
TBL
     [,1] [,2] [,3] [,4] [,5]
[1,] 8278 2183 3844  390 2341
[2,] 1659  422  899  137  662

chisq.test(TBL)

        Pearson's Chi-squared test

data:  TBL
X-squared = 75.376, df = 4, p-value = 1.659e-15

Pearson residuals (the squares of which sum to give the chi-squared statistic) are as shown below. Looking at the residuals with the largest absolute values often points the way to interesting differences. It seems that the first two columns (Prx and AXJ) have lowest proportions of women and that the last two (CPX and AXJ) are a little nearer 50:50.

 chisq.test(TBL)$resi
          [,1]      [,2]       [,3]      [,4]      [,5]
[1,]  1.608733  1.103264 -0.6082985 -1.989678 -2.355970
[2,] -3.415697 -2.342474  1.2915527  4.224527  5.002247

It might be worthwhile to do a chi-squared test that omits column 3 (where the ratio is closer to the overall ratio). The disparity is also overwhelmingly significant with a P-value near $0.$

chisq.test(TBL[, c(1,2,4,5)])

        Pearson's Chi-squared test

data:  TBL[, c(1, 2, 4, 5)]
X-squared = 73.482, df = 3, p-value = 7.664e-16

What needs further examination and (perhaps) formal testing depends on your purpose for doing this study. Are you surprised that there are more male than female authors? Did you expect that the degree of disparity would differ markedly by journal or journal type? An unmotivated list of tiny P-values is not going to lead to interesting or useful conclusions.