# Post hoc test for chi-squared with given (unequal) probabilities

I am conducting a chi-squared test to see whether the categories survey respondents mention match the distribution of categories in the population. This test is significant, but I am wondering how to run post hoc tests to determine which of the categories significantly deviates from the known probability.

responses <- matrix(c(1247,1362, 184, 224), ncol=4)
colnames(responses) <- c("Cat 1", "Cat 2", "Cat 3", "Cat 4")
responses<-as.table(responses)
proportions <- c(0.39, 0.40, 0.10, 0.11)
chisq.test(responses, p = proportions)


Lukas

There are some ideas at this Cross Validated post

I believe looking at the standardized residuals from the analysis is appropriate in the case of chi-square goodness-of-fit test with unequal theoretical proportions. A standardized residual >1.96 or <-1.96 indicates an interesting observation corresponding to the alpha = 0.05 level.

A problem with this approach with your data is that your sample size is large enough that relatively small differences between the theoretical and observed proportions are likely to be significant. The logical approach here, then, is to report the differences in proportions in some way, and assess whether these differences are large enough to be practically meaningful.

responses <- matrix(c(1247,1362, 184, 224), ncol=4)
colnames(responses) <- c("Cat 1", "Cat 2", "Cat 3", "Cat 4")
responses<-as.table(responses)
proportions <- c(0.39, 0.40, 0.10, 0.11)
chisq.test(responses, p = proportions)

Test = chisq.test(responses, p = proportions)

StdRes = Test\$stdres
names(StdRes) = colnames(responses)
StdRes

###     Cat 1     Cat 2     Cat 3     Cat 4
###  2.626653  5.767644 -7.142785 -6.276560

Pval = 2 * pnorm(abs(StdRes), lower.tail=F)
Pval

###        Cat 1        Cat 2        Cat 3        Cat 4
### 8.622926e-03 8.038742e-09 9.145890e-13 3.461461e-10

observed.prop = responses / sum(responses)
XT = rbind(proportions, observed.prop)
rownames(XT)[1]="Theoretical"
rownames(XT)[2]="Observed"
XT

###                Cat 1     Cat 2      Cat 3      Cat 4
### Theoretical 0.3900000 0.4000000 0.10000000 0.11000000
### Observed    0.4133245 0.4514418 0.06098774 0.07424594

barplot(XT,
beside = T,
xlab   = "Group",
col    = c("cornflowerblue","blue"),
legend = rownames(XT))


• Thank you, that is very helpful! I completely agree that substantive interpretations matter more with this sample size, but the journal I am submitting to wants p-values as well ... – Lukas Wallrich Mar 9 at 19:15