I have an experiment with two groups under different conditions where a single dependent variable is measured repeatedly at multiple times. It seems the right parametric test to use here is two-factor mixed ANOVA:

"A mixed ANOVA compares the mean differences between groups that have been split on two "factors" (also known as independent variables), where one factor is a "within-subjects" factor and the other factor is a "between-subjects" factor."

However, my data is strongly non-normal (and some of the dependent variables are ordinal anyway).

What is a non-parametric equivalent test to two-factor mixed ANOVA?

  • $\begingroup$ So to clarify: you also need to predict multiple output variables? $\endgroup$ – David Ernst Sep 27 '17 at 1:13
  • 2
    $\begingroup$ Are you looking for something like this stats.stackexchange.com/questions/12151/… ? $\endgroup$ – Karel Macek Sep 27 '17 at 2:31
  • $\begingroup$ @DavidErnst Yes, but not necessarily at the same time; I think I can do one output variable at a time. Is it necessarily to ANOVA them together somehow? $\endgroup$ – Alex I Sep 27 '17 at 6:20
  • $\begingroup$ @KarelMacek Yes, I think that is exactly it, except I'm not completely sure how to do some of the things the top answer (permutation test) talks about in practice. How do I do "full and reduced model test"? $\endgroup$ – Alex I Sep 27 '17 at 6:33

One of the standard techniques in such situations are due to Brunner and Langer [1]. These non-parametric mixed-effects models can deal with multiple within-subject factors and some between subject factor. In some fields (e.g. dental medicine), there are very popular. In R, they are implemented in the nparLD package.

[1] Brunner E, Domhof S, Langer F (2002). Nonparametric Analysis of longitudinal Data in Factorial Experiments. John Wiley & Sons, New York.


Considering the categorical variable, the design matrix for the explanatory variables should be adjusted such that the design is estimable.

On the other hand, for the non-normal error distribution, there are options for you before redirecting to nonparametric methods.

1) You can depict and measure the amount of nonnormality of your data. The classical F test is robust to nonnormality to some extent. If the underlying distribution is reasonably symmetric you can feel safe. If you observe significant skewness on the error distribution and do not feel safe with the F result, proceed to the next option.

2) I suggest using robust ANOVA methods. Please read through the following paper. I personally implemented Tiku's MML method which is robust under normality and is also flexible such that you can incorporate nonnormal distributions in the Maximum Likelihood context.


3) If neither of the above works for you, for a straightforward quick solution, you can try data transformation and nonparametrics selectively.

Good Luck!


ezPerm function from ez package provides permutation-based versions of different ANOVAs, including mixed. It does not assume any normality.

  • $\begingroup$ Thanks, this looks good. Is there a paper or book that describes permutation-based ANOVA? $\endgroup$ – Alex I Oct 2 '17 at 18:34
  • 1
    $\begingroup$ I think most of the papers that use ez cite its documentation directly (scholar.google.ru/scholar?cluster=310992833082440893), and the permutation ANOVA itself is described in Pesarin, Salmaso. Permutation Tests for Complex Data (2010) and Bonnini, Corain, Marozzi, Salmaso. Nonparametric Hypothesis Testing - Rank and Permutation Methods with Applications in R (2014) $\endgroup$ – Evgeniy Riabenko Oct 2 '17 at 19:47
  • $\begingroup$ From the docs: "ezPerm is a work in progress. Under the current implementation, only main effects may be trusted". While the general method looks good, I'm not sure I can use the software package. $\endgroup$ – Alex I Oct 5 '17 at 3:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.