I've been searching on the internet and books for quite a long time now and have come to the conclusion that normality of the residuals (each values minus the mean of each group) is the same as looking for normality of the distribution of y in each group separately. ( Normality of dependent variable = normality of residuals? ) However, when I am doing both og these methods with my dataset, the method with the residuals give me a very different answer than the method for normality of Y in each group. For instance, I find with the residual method (only one graph) that the age and height are not normally distributed but that the waist circumference is. Looking at the distribution of Y in each group I find that the age and height are normally distributed in each of the 4 groups but that the waist circumference is only normally distributed in group 1, 2 and 4. Not in group 3. How do I interpret these results? Which method should I use? I want to use ANOVA and p-values so I need my data to be normally distributed. I have 4 groups with 20 datapoint in each group.
-
3$\begingroup$ Keep in mind that if you look at enough subgroups, you'll eventually find some that don't look normal, even if they're all drawn from normal populations. [On the other hand, in reality probably none of them are actually normal in practice.] $\endgroup$– Glen_bJul 20, 2016 at 9:03
-
$\begingroup$ Thank you for the answer! Do you mean that I should focus on the residuals since it is, here, only one test for the whole distribution over all the groups, and not focus on the normality of the distribution of Y in each og the groups since the type I error here is higher? How would you then explain that some og the distributions are not normal using the residuals but appear normal when looking at the distribution of Y in each group? Isn't the sample here large enough to show that the distribution is normal when looking at the groups separately? $\endgroup$– user123867Jul 20, 2016 at 9:28
-
$\begingroup$ Many questions! 1. My comment was not an answer, just something to keep in mind. 2. I would not advise testing at all, since hypothesis tests will tend to reject when there may truly be nothing to worry about (your data are drawn from a distribution that's quite close to normal when you have a large sample size) while failing to reject when there's a problem (the data are drawn from a non-normal distribution that is of a kind that will cause problems for your inference ... but the test will still fail to reject). The problem is closer to an effect-size issue than one of hypothesis testing..ctd $\endgroup$– Glen_bJul 20, 2016 at 9:41
-
$\begingroup$ ctd... 3. No matter how large your sample (a million points, a billion points...) you cannot say your data are drawn from a normal distribution -- nothing can take a sample and tell you that the population they came from was normal. 4. It's easily possible to reject normality in one subgroup while failing to reject when all the subgroups are together. This can happen when all of the subgroups really do come from a normal distribution and it can happen when none of the subgroups come from a normal distribution (examples of both cases are easy to construct). $\endgroup$– Glen_bJul 20, 2016 at 9:44
-
1$\begingroup$ ctd... The critical issue here is how much does whatever non-normality you might have impact your inference - how badly non-normal is it and how sensitive is what you're doing to that (at least to that kind of non-normality). Since you only have 20 points in each group I'd be inclined to look at (not test) the combined residuals (e.g. in a Q-Q plot) to try to asses "how non-normal" they might be and in what particular way. [If your subgroups were larger you might consider them separately] $\endgroup$– Glen_bJul 20, 2016 at 9:51
1 Answer
Not knowing which methods you used to test for the normality of the residuals and the dependent variable, respectively, it's difficult for me to give you an exact answer. However, I assume that you used a visual comparison or some kind of significance test to check for normality.
Since you mentioned that you only have 20 datapoints per group, I think that the problem lies with the sample size. If you use our standard "off-the-shelf" Frequentist test to assess the normality of a group. For example, if you use the Shapiro-Wilk test to check for normality, you are essentially comparing your (standardised and ordered) sample with an ordered sample drawn from a standard normal distribution. If your sample deviates too much from the standard normal distribution, the difference is deemed "significant" (for example on the 0.05 level), giving you a hint that your sample should not be regarded as normally distributed.
But the Shapiro-Wilk test, like most normality test, is highly susceptible to changes in the sample size. If your sample size is too low, it is very hard to detect a difference to a normally distributed sample so most of the test results will be non-significant. If you increase the sample size, however, even small deviations from the normal distributions will turn out to be "significant".
This is probably what happened in your case. When you were using the residuals to test for normality, you had a total of 420 = 80 data points for each variable (height, weight & waist circumference), with the result that two out of three tests turned out to be significant. When you were using the data points within each group, you conducted more tests in total (43 = 12 tests for each variable in each group), but due to the low sample size in each test only 1 out of 12 tests turned out to be significant (waist circumference in group 3)
I hope that helped to clear things up a bit. In order to give you any meaningful recommendation on which methods to use, you need to present more information on your data set and the exact tests you used.
-
$\begingroup$ Thank you for responding. I did use the Shapiro-Wilk test since I find it difficult to use a cut off when looking at graphs. I do now understand why using the residual can show non-normality while using datapoint in each group do not show this (since as you are saying, the dataset is smaller). However I do not understand why the datapoints in each group can show non normality when the residuals do not show this (the sample size is larger). Is this due to some random error? Which method should I use? Random error or datapoints in each group, there is only little overlap between these methods. $\endgroup$– user123867Jul 20, 2016 at 10:28
-
$\begingroup$ I also find it difficult to understand why people say that using these two methods are exactly the same when I find such different answers! (theanalysisfactor.com/checking-normality-anova-model) (stats.stackexchange.com/questions/60410/…) $\endgroup$– user123867Jul 20, 2016 at 10:37
-
$\begingroup$ I'll address your questions one by one 1) "I do not understand why the datapoints in each group can show non normality when the residuals do not show this?" Where did this happen? As I understand it from the example above, the following cases arose from your tests: $\endgroup$– studentJul 20, 2016 at 12:59
-
$\begingroup$ A) the normality tests for age & height were significant in the residual-based test, but non-significant in the group-based tests B) waist-circumference was non-significant in the residual based test and in the group-based test. Remember: "significant" means "significant deviation from normality" aka "non-normal" (only in this test scenario, of course) These resuls are perfectly in line with the reasoning above. The only exception is the signficant result for group 3, but that might be a [Multiple Comparisons Problem] (en.wikipedia.org/wiki/Multiple_comparisons_problem) $\endgroup$– studentJul 20, 2016 at 13:08
-
$\begingroup$ 2) Which method should I use? If you really need to rule out any non-normality, then I would go for the residual based method since it will give you more power to detect potential deviations from normality. However, as @Glen_b explained above you first need to consider how important normality is for your testing scenario. $\endgroup$– studentJul 20, 2016 at 13:16