# Multiple comparisons for not normal and heterogeneous data

I have a data set that has 3 groups and a single variable. Under normal circumstances, a one-way ANOVA would be used for comparisons between my three groups. HOWEVER I ran a Shapiro-Wilk test for normality (I tried to normalize the data but it was unsuccessful) and Levene's test for homogeneity and turns out my data is both: not normally distributed and heterogeneous.

What statistical method should I run instead of a One-way ANOVA and what post-hoc test may be better?

Edit: Below is a picture of the general data that I am referring to. Data is skewed but upon visual inspection doesn't look incredibly heterogeneous, despite the Levene's test of homogeneity results.

• Those tests are surprisingly unhelpful for your purposes. Shapiro-Wilk is going to reject an almost true null hypothesis once the sample size gets to be large; Levene is going to reject an almost true null hypothesis once the sample size gets to be large. By "an almost true null hypothesis", I mean a null hypothesis that, while false, is sufficiently close to true that subsequent testing is likely to be robust to such minor violations. The classic example is letting the central limit theorem take care of non-normal data. When you look at your data, do the assumptions seem very violated?
– Dave
Commented Jul 23, 2020 at 21:28
• Thank you for your response!. I have updated the post to include a picture of the data for my three groups. I believe the data is not normally distributed, which would be consistent with Shapiro-Wilk test. However, I don't fully believe that my data among groups is too heterogeneus, which would go against Levene's results. Commented Jul 24, 2020 at 20:20

Firstly, I'm not sure whether the extent of non-normality you see is really concerning - assuming this some kind of randomized experiment - for the valditity of methods assuming normal residuals. Very often, one may have some violation of the assumption of normal regression resdiuals without this mattering one bit for whether a linear model with normal residuals gives very sensible answers (there is an extensive literature on this).

Secondly, might there be other covariates (other than the group) that explain the distribution that you should include the model, after which model residuals might be reasonably normal.

Finally, this looks a bit like a log-transformation might be helpful. Even if some test for normality may still complain you might come closer to a normal distribution (however, note that this changes the interpretation of effects afer back-transformation from additive to multiplicative).

The most frequently used alternative to the traditional F-test ANOVA in your circumstance is the Kruskal-Wallis one-way ANOVA. If a typical ANOVA would have been a good choice under the assumption of normality, then it’s very likely the Kruskal-Wallis test assumptions are satisfied.

Note that the two do not formally measure the same thing so adjustments will need fo be made to your hypothesis statements. As always, make sure to take a close look at the limitations of the test and circumstances for which an ANOVA might not be suitable at all!

This test should be paired with the Dunn Test post-hoc. Implementations for both are available in most statistical computing software, although some care should be taken that the Dunn test has been implemented with attention to its use as a post-hoc test for Kruskal-Wallis.

If these do not appear satisfactory, you can explore other options by looking for tests that fall under the category of “non-parametric,” as those are most likely to not require an underlying normal distribution. Some resources for which can be found here..

In future questions, it is useful if you were to provide some additional details on the characteristics of the variable you have measured and what hypothesis you intend to test. This let’s responders be more confident ahem that they have lead you in the right direction!

• Thank you so much for your response! Doing Kruskal-Wallis followed by a Dunn's test with Bonferroni's correction was my first instinct but then I realized my data was not heterogeneous and I was worried that may affect the outcomes of the test. Those resources you shared are really helpful. And thank you for your recommendation on adding some of the data to have a better general picture. Commented Jul 24, 2020 at 20:01
• I have just added a picture to the original post. Thanks again! Commented Jul 24, 2020 at 20:21