2
$\begingroup$

In testing for the normality assumption for 2 way ANOVA, the question whether normality within cells implies normality of residuals has been asked before and I have found so far different answers from different forums and still it's not clear to me.

I have the data of 6 patients that have been treated twice with 2 different drugs and each with 3 different levels of dose. So there are 2 drugs with 3 levels and I end up with six cells.

Doing 2 way anova I get a significant interaction and also the data passes the homoscedasticity test (Levene test). The normality assumption is true when is tested within each cell (6 cells were tested separately). However, when I do the test over the residuals of the anova model the result is the opposite (p<0.01).

Can someone please explain which of these two normality tests should I rely on (over residuals or within each cell) and if the normality of residuals matters.

How can I confirm that I am allowed to run 2 way ANOVA? Does this difference stem from the low sample size of my population?

Improve this question
$\endgroup$
13
  • 2
    $\begingroup$ Welcome to our site. Could you please direct us to the "different answers" you have found (assuming they appeared somewhere on CV) so we might be able to improve them? $\endgroup$
    – whuber ♦
    Aug 31, 2018 at 13:07
  • $\begingroup$ Your test within cells are likely to have lower power if you did six separate tests so your results then would not be in conflict. Can you edit your question to clarify if that is what you did? $\endgroup$
    – mdewey
    Aug 31, 2018 at 13:12
  • $\begingroup$ @mdewey: 6 separate tests (already edited). no conflict means that I should rely on residuals? $\endgroup$
    – Ali Nouri
    Aug 31, 2018 at 13:18
  • 1
    $\begingroup$ If you regard 6 patients as your population, how do you interpret significance test results? $\endgroup$
    – Nick Cox
    Aug 31, 2018 at 13:33
  • $\begingroup$ @NickCox: Unfortunately at the moment thats the max number I can have. But with these limitations is there anyway to proceed? $\endgroup$
    – Ali Nouri
    Aug 31, 2018 at 13:38

2 Answers 2

Reset to default
2
$\begingroup$

Well, that depends on what you want to do with the results.

From this: "...in ANOVA, you actually have two options for testing normality. If there really are many values of Y for each value of X (each group), and there really are only a few groups (say, four or fewer), go ahead and check normality separately for each group.

But if you have many groups (a 2x2x3 ANOVA has 12 groups) or if there are few observations per group (it’s hard to check normality on only 20 data points), it’s often easier to just use the residuals and check them all together."

Which means that 1) If you have less than 20 in each cell, the power for checking normality in each cell is too low to be strongly indicative of residual structure. 2) You can do ANOVA but the results may be wonky. Some suggest not testing for normality because mean values from ANOVA of data are not very sensitive to non-normality. Non normal extreme values, "outliers", and skewness are often more important than non-normality itself.

An alternative may be to rank the entire data, but that depends on the data all having the same scale. So, what does the data look like? Ranking will generally normalize the residuals, if appropriate, and make a non-parametric ANOVA possible. There are also a number of non-parametric alternatives to ANOVA. When in doubt, do a non-parametric test as well as parametric ANOVA and compare them.

BTW, there are more flavors of ANOVA than of Baskin-Robins ice cream, so it depends somewhat on what information you are attempting to extract. For a brief introduction (SPSS) of ANOVA assumptions see this link.

Improve this answer
$\endgroup$
1
$\begingroup$

Welcome to CV.

Certainly @Carl's answer raises good points. One clear problem with all tests of normality is that the p value depends partly on sample size while the assumption of normality does not. That is, the normality assumption is not that the deviation from normality is not significant, it is that the errors are normal.

In the old days, before fast computers, it made sense to try to assess normality because the methods that don't assume normality were hard to do. Now, they are not.

So, I suggest doing (say) robust regression or quantile regression and comparing the results with your ANOVA results. If they are similar, all is well. If they are quite different, then you have to think.

Improve this answer
$\endgroup$
6
  • $\begingroup$ If this is a "clear problem" with tests of normality then it's a problem with all hypothesis tests! Thus, I doubt your text expresses the idea you intended to get across. $\endgroup$
    – whuber ♦
    Sep 2, 2018 at 13:30
  • 1
    $\begingroup$ @whuber There may be a problem with hypothesis testing for normality for small values of $n$ and for very large values of $n$, or at least that claim has been put forward by multiple authors. The question you raise may be worth asking separately, i.e., exploring. $\endgroup$
    – Carl
    Sep 3, 2018 at 15:30
  • 1
    $\begingroup$ (+1) For participating and saying something useful. $\endgroup$
    – Carl
    Sep 3, 2018 at 15:32
  • 1
    $\begingroup$ @whuber There certainly are problems with hypothesis testing in general. As you are surely aware, these have been extensively discussed for the past decade at least. This probably isn't the place to go into the general question. For this specific case, significance is pretty much completely irrelevant. The real question is how severely the assumptions are violated. $\endgroup$
    – Peter Flom
    Sep 3, 2018 at 23:15
  • 1
    $\begingroup$ @whuber In the ANOVA context it may be that NHST is too conservative. For example, if the residuals are Student's-t distributed with $df\geq 2$, I do not think that in that case any ANOVA non-normality would be particularly relevant. $\endgroup$
    – Carl
    Sep 6, 2018 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.