5
$\begingroup$

I have several dependant variables that are non normal in distribution: Kolmogorov-Smirnov test is significant, skewness ranges up to 8 for some variable, and kurtosis is generally about 2 but in a couple of cases raises to 12! It is a repeated-measures (RM) ANOVA model for the normally distributed variables.

A couple of questions:

  1. How robust is RM ANOVA to violations of normality, when there is an equal number of observations per group?
  2. If I use non-parametric tests, do I need to correct for multiple testing that impacts type 1 error?
Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ Please clarify: are the DVs themselves non-normal or are their residuals non-normal? Only the latter is of any concern. $\endgroup$
    – whuber ♦
    Mar 29, 2012 at 15:28
  • $\begingroup$ Aah _ i dont know - the distribution of the scores - how do i tell if residuals are non normal? $\endgroup$
    – Sheila
    Mar 29, 2012 at 16:07
  • 1
    $\begingroup$ It's time to do a little research. $\endgroup$
    – whuber ♦
    Mar 29, 2012 at 17:04
  • 1
    $\begingroup$ Ok - done that and found that the residuals are not normally distributed so back to original question please $\endgroup$
    – Sheila
    Mar 29, 2012 at 17:10
  • $\begingroup$ May be multivariate normality (link2, link) should be checked in the case of RM ANOVA? $\endgroup$
    – abc
    Mar 30, 2012 at 17:19

3 Answers 3

Reset to default
9
$\begingroup$

Don't look at it as a binary thing: "either I can trust the results or I can't." Look at it as a spectrum. With all assumptions perfectly satisfied (including the in most cases crucial one of random sampling), statistics such as F- and p-values will allow you to make accurate sample-to-population inferences. The farther one gets from that situation, the more skeptical one should be about such results. You've got a substantial degree of nonnormality; that's one strike against accuracy. Now how about the other assumptions underlying the use of ANOVA? Size it all up the best you can, and document in a footnote or a technical section what you find. You also should look at this page, as @William pointed out.

As to your last question, I don't believe you need to change your strategy vis-a-vis multiple comparisons just because you move from a parametric to a nonparametric test. If you want to describe the rationale for your current approach, I'm sure people will be glad to comment on it.

Improve this answer
$\endgroup$
4
$\begingroup$

Let me state a couple of things. First, I think it's best to understand repeated measures ANOVA as actually a multi-level model in disguise, and that may create additional complexities here. I should let one of CV's contributors who are more expert on multi-level models address that issue.

However, in general, it's worth noting that not all assumptions are created equal. People tend to think that the normality assumption is vital, whereas I think of it as the least important. Heterogeneity is a bigger deal. Skew is potentially more damaging than kurtosis, but if it isn't too large, and all groups are skewed in the same direction, it may not be lethal. Basically, whether or not the residuals are normal has to do with whether the p-values are accurate, but the parameter estimates should remain unbiased. On the other hand, heterogeneity of variance has to do with the efficiency of the OLS estimator.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ @whuber gave a nice explanation of why and how the normality assumption really matters in ANOVA testing. $\endgroup$
    – chl
    Mar 30, 2012 at 21:20
  • $\begingroup$ @chl, thanks for the link--clear, concise & informative, as always $\endgroup$
    – gung - Reinstate Monica
    Mar 30, 2012 at 22:01
2
$\begingroup$

My understanding is that ANOVA including repeated measures is robust to violations to normality of errors assumptions. However there is indications that the errors should be equal in their variation across different factor levels.

Can I trust ANOVA results for a non-normally distributed DV?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.