0
$\begingroup$

I am conducting a chi-squared test to see whether the categories survey respondents mention match the distribution of categories in the population. This test is significant, but I am wondering how to run post hoc tests to determine which of the categories significantly deviates from the known probability.

responses <- matrix(c(1247,1362, 184, 224), ncol=4)
colnames(responses) <- c("Cat 1", "Cat 2", "Cat 3", "Cat 4")
responses<-as.table(responses)
proportions <- c(0.39, 0.40, 0.10, 0.11)
chisq.test(responses, p = proportions)

Many thanks in advance.

Lukas

$\endgroup$
0
$\begingroup$

There are some ideas at this Cross Validated post

I believe looking at the standardized residuals from the analysis is appropriate in the case of chi-square goodness-of-fit test with unequal theoretical proportions. A standardized residual >1.96 or <-1.96 indicates an interesting observation corresponding to the alpha = 0.05 level.

A problem with this approach with your data is that your sample size is large enough that relatively small differences between the theoretical and observed proportions are likely to be significant. The logical approach here, then, is to report the differences in proportions in some way, and assess whether these differences are large enough to be practically meaningful.

responses <- matrix(c(1247,1362, 184, 224), ncol=4)
colnames(responses) <- c("Cat 1", "Cat 2", "Cat 3", "Cat 4")
responses<-as.table(responses)
proportions <- c(0.39, 0.40, 0.10, 0.11)
chisq.test(responses, p = proportions)

Test = chisq.test(responses, p = proportions)

StdRes = Test$stdres
names(StdRes) = colnames(responses)
StdRes

   ###     Cat 1     Cat 2     Cat 3     Cat 4 
   ###  2.626653  5.767644 -7.142785 -6.276560 

Pval = 2 * pnorm(abs(StdRes), lower.tail=F)
Pval

   ###        Cat 1        Cat 2        Cat 3        Cat 4 
   ### 8.622926e-03 8.038742e-09 9.145890e-13 3.461461e-10 

observed.prop = responses / sum(responses)
XT = rbind(proportions, observed.prop)
rownames(XT)[1]="Theoretical"
rownames(XT)[2]="Observed"
XT

   ###                Cat 1     Cat 2      Cat 3      Cat 4
   ### Theoretical 0.3900000 0.4000000 0.10000000 0.11000000
   ### Observed    0.4133245 0.4514418 0.06098774 0.07424594

barplot(XT,
        beside = T,
        xlab   = "Group",
        col    = c("cornflowerblue","blue"),
        legend = rownames(XT))

enter image description here

$\endgroup$
  • $\begingroup$ Thank you, that is very helpful! I completely agree that substantive interpretations matter more with this sample size, but the journal I am submitting to wants p-values as well ... $\endgroup$ – Lukas Wallrich Mar 9 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.