19
$\begingroup$

If the model does not satisfy ANOVA assumptions (normality in particular), if one-way, Kruskal-Wallis non-parametric test is recommended. But, what if you have multiple factors?

$\endgroup$
15
$\begingroup$

You can use a permutation test.

Form your hypothesis as a full and reduced model test and using the original data compute the F-statistic for the full and reduced model test (or another stat of interest).

Now compute the fitted values and residuals for the reduced model, then randomly permute the residuals and add them back to the fitted values, now do the full and reduced test on the permuted dataset and save the F-statistic (or other). Repeate this many times (like 1999).

The p-value is then the proportion of the statistics that are greater than or equal to the original statistic.

This can be used to test interactions or groups of terms including interactions.

$\endgroup$
  • 5
    $\begingroup$ For a discussion of different permutation strategies in factorial ANOVA-designs, see e.g. avesbiodiv.mncn.csic.es/estadistica/permut1.pdf (pdf) $\endgroup$ – caracal Jun 21 '11 at 11:40
  • 3
    $\begingroup$ This works, but what happens to the power of the test? E.g., even if there is only one (far) outlying value and the rest of the residuals are normally distributed, it appears that using the F-statistic may have little power in the permutation test to detect anything. The paper referenced by @caracal discusses the subtleties and assesses when the F-statistic approach works and when it might fail. $\endgroup$ – whuber Jun 21 '11 at 14:31
  • $\begingroup$ "The p-value is then the proportion of the statistics that are greater than or equal to the original statistic" --> to the original statistic calculated on the full model. correct? $\endgroup$ – Yannick Wurm Nov 8 '11 at 2:57
  • 1
    $\begingroup$ @toto_tico, using ranks is one option for non-parametric tests, but is not the only one (permutation testing is another that does not rely on ranks). Combining factors into a single factor works if you want to test all-or-nothing, but does not work for testing if the interaction is significant beyond the effects of the main effects, or testing one factor given the other factor is in the model. $\endgroup$ – Greg Snow Nov 13 '15 at 16:57
  • 1
    $\begingroup$ @toto_tico, just code it directly. See the example I added based on your other comment (stats.stackexchange.com/questions/41199/…). $\endgroup$ – Greg Snow Nov 16 '15 at 17:33
13
$\begingroup$

The Kruskal-Wallis test is a special case of the proportional odds model. You can use the proportional odds model to model multiple factors, adjust for covariates, etc.

$\endgroup$
  • 3
    $\begingroup$ If one would like to learn more about the connection between K-W and the proportional odds model, what would be a good reference? $\endgroup$ – whuber Jun 21 '11 at 14:24
  • 5
    $\begingroup$ @ARTICLE{pet89ord, author = {Peterson, Bercedis}, year = 1989, title = {Re: {Ordinal} regression models for epidemiologic data}, journal = Am J Epi, volume = 129, pages = {745-748}, annote = {proportional odds model; partial proportional odds} } @ARTICLE{mcc80reg, author = {{McCullagh}, Peter}, year = 1980, title = {Regression models for ordinal data}, journal = JRSSB, volume = 42, pages = {109-142}, annote = {ordinal logistic model} }See also Whitehead Stat in Med 1993 p. 2257 $\endgroup$ – Frank Harrell Jun 22 '11 at 2:36
7
$\begingroup$

Friedman's test provides a non-parametric equivalent to a one-way ANOVA with a blocking factor, but can't do anything more complex than this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.