14
$\begingroup$

I have cross classified data in a 2 x 2 x 6 table. Let's call the dimensions response, A and B. I fit a logistic regression to the data with the model response ~ A * B. An analysis of deviance of that model says that both terms and their interaction are significant.

However, looking at the proportions of the data, it looks like only 2 or so levels of B are responsible for these significant effects. I would like to test to see which levels are the culprits. Right now, my approach is to perform 6 chi-squared tests on 2 x 2 tables of response ~ A, and then to adjust the p-values from those tests for multiple comparisons (using the Holm adjustment).

My question is whether there is a better approach to this problem. Is there a more principled modeling approach, or multiple chi-squared test comparison approach?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I once asked the same question on the R mailing list, and didn't get a response. I'd suggest you to change your title since your question is regarding "post hoc analysis of chi square - to detect the cause of the significance" (a shorter titles then the one I proposed would be better :) ) $\endgroup$
    – Tal Galili
    Aug 2, 2010 at 20:33
  • 1
    $\begingroup$ Just look at the betas for your culprits.... And use a poisson, log-linear model. You then get the same thing as what the chi-square test gives you, but you get all of the different tests at once. $\endgroup$
    – probabilityislogic
    Jan 11, 2012 at 20:46

3 Answers 3

Reset to default
12
$\begingroup$

You should look into "partitioning chi-squared". This is similar in logic to performing post-hoc tests in ANOVA. It will allow you to determine whether your significant overall test is primarily attributable to differences in particular categories or groups of categories.

A quick google turned up this presentation, which at the end discusses methods for partitioning chi-squared.

http://www.ed.uiuc.edu/courses/EdPsy490AT/lectures/2way_chi-ha-online.pdf

Improve this answer
$\endgroup$
4
  • $\begingroup$ Interesting. Did you ever come across an R implementation of this ? $\endgroup$
    – Tal Galili
    Aug 4, 2010 at 3:16
  • $\begingroup$ No, not directly. However, R will give you everything you need to do this--such as: the observed counts, the expected values, and the residuals for each cell. x <- matrix(c(12, 5, 7, 7), ncol = 2) chisq.test(x)$expected chisq.test(x)$observed chisq.test(x)$residuals $\endgroup$
    – Brett
    Aug 4, 2010 at 15:55
  • $\begingroup$ I'll give you the tick, since this should be useful for my research life. However, this approach is applicable to an i x j matrix. However, my question involves an i x j x k matrix, $\endgroup$
    – JoFrhwld
    Aug 4, 2010 at 17:02
  • 2
    $\begingroup$ Chi-square partitioning is extensible to multi-way contingency tables. Here's the article that Agresti cites in his book, in fact... H. O. Lancaster (1951) "Complex Contingency Tables Treated by the Partition of χ2" Journal of the Royal Statistical Society. Series B (Methodological), Vol. 13, No. 2 $\endgroup$
    – Brett
    Aug 4, 2010 at 18:13
1
$\begingroup$

The unprincipled approach is to discard the disproportionate data, refit the model and see if logit/conditional odds ratios for response and A are very different (controlling for B). This might tell you if there's cause for concern. Pooling the levels of B is another approach. On more principled lines, If you're worried about relative proportions inducing Simpson's paradox, then you can look into the conditional and marginal odds ratios for response/A and see if they reverse.

For avoiding multiple comparisons in particular, the only thing that occurs to me is to use a hierarchical model which accounts for random effects across levels.

Improve this answer
$\endgroup$
0
$\begingroup$

Post Hoc test may fit to your problem. chisqPostHoc() function in R tests for significant differences among all pairs of populations in a chi-square test. Even though, I haven't use it but this link may be useful. https://www.rforge.net/doc/packages/NCStats/chisqPostHoc.html

Another alternative may be chisq.desc() function from EnQuireR package.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The link is dead! $\endgroup$
    – kjetil b halvorsen ♦
    Aug 14, 2022 at 13:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.