9
$\begingroup$

The Marascuilo procedure as described here seems to be a test that addresses the issue of multiple comparisons for proportions when you want to test which specific proportions are different from each other after rejecting the null in an overall chi-square test.

However, I am not very familiar with this test. So, my questions:

  1. What nuances (if any) should I worry about when using this test?

  2. I know of at least two other approaches (see below) to address the same issue. Which test is the 'better' approach?:

$\endgroup$
  • 1
    $\begingroup$ Perhaps this discussion is relevant as -- it isn't often used because it is very conservative (much like Scheffe's Method)? $\endgroup$ – M. Tibbits Oct 12 '10 at 19:06
  • 3
    $\begingroup$ Surely you mean "after rejecting the null" not "after failing to reject the null"? And it seems there's only one L in 'Marascuilo' (NIST's error, not yours): Leonard A. Marascuilo. Large-sample multiple comparisons. Psychological Bulletin, 1966; 65(5): 280-290. dx.doi.org/10.1037/h0023189. $\endgroup$ – onestop Oct 12 '10 at 21:58
7
$\begingroup$

Just a partial answer because I've never heard of this method. From what I read in the link you provided, it seems to be a single-step procedure (much like Bonferroni, except we rework the test statistics instead of the p-value) which is likely to be too conservative.

In R, there is a function pairwise.prop.test() which allows any correction for multiple comparisons (single-step or step-down FWER methods or FDR-based), but it is quit what you already suggested (although Bonferroni is by far too conservative, but still very used in practice). A resampling approach, using permutation, might be interesting too. The coin R package provides a well-established testing framework in this respect, see §5 of Implementing a Class of Permutation Tests: The coin Package, but I never had to deal with permutation tests on categorical data in a post-hoc way.

About the analysis of subdivided contingency tables, I generally consider specific associations as a guide to develop additional hypotheses (as for any unplanned comparisons), but this is another question. I generally just use visualization tools, like mosaicplot from Michael Friendly, Pearson's residuals, and if I seek to explain specific patterns of association I use log-linear models.

$\endgroup$
  • $\begingroup$ Thanks for the pointers to R packages/functions. I will take a look at them. $\endgroup$ – user28 Oct 12 '10 at 22:40
2
$\begingroup$

I would like to see the Marascuilo procedure used more often. Quite frequently I see people calculating the chi-square on a subset of the main table ie two categories at the time but without actually doing the partitioning correctly. The reason why they do it this way as far as iI understood is that they can't bear grouping the categories as that will make the interpretation really hard. At the end of the day it depends of the audience as well coz if they don't know it they might just recommend the usual Bonferroni approach

$\endgroup$
  • 1
    $\begingroup$ Would you mind explaining why such procedure should be preferred? $\endgroup$ – chl Oct 13 '10 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy